1
0
Fork 0
mirror of https://github.com/ansible-collections/community.general.git synced 2024-09-14 20:13:21 +02:00
Commit graph

2546 commits

Author SHA1 Message Date
Brian Coca
e7941b0d4e
avoid chroot paths (#32778)
* avoid chroot paths

fixes #32764

* check name
2017-11-10 13:26:49 -05:00
Sloane Hertel
33a704cc61 Add documentation for the aws_ssm lookup plugin. (#32763) 2017-11-10 12:15:19 -05:00
Matt Davis
5ff36c3423 Enable autoloading of inventory plugins (#32709)
* Automatically loads and executes an inventory plugin specified by a standard YAML inventory config file containing a `plugin` key at its root.
* Moved inventory PluginLoader to a shared global instance.
2017-11-09 19:38:34 -05:00
Brian Coca
59d5481abb
fixes to events/callbacks (#32633)
* fixes to events/callbacks

  - made note of 'not called' methods for future fixes
  - removed uncalled v2_runner_on_file_diff because dupe of v2_on_file_diff, which is called
  - removed v2_runner_on_no_hosts due to existing pb level ones, which are called
  - removed v2_on_setup, it is just a task, triggers normal task events
  - v2_on_notify is now called when a handler is notified
  - TODO: async, cleanup? and import events
    these currently occur in code that has no access to sending events

* corrected display
2017-11-09 17:12:37 -05:00
Brian Coca
0e74731d73 Added callback that removes callback noise
for those that only want logging but no events to screen
2017-11-09 17:12:13 -05:00
u625030
6d597ac05a Update win_copy for #32677 (#32682)
* Update win_copy for #32677

enable large zip file support in win_copy

* Update win_copy.py
2017-11-10 06:34:29 +10:00
Nathaniel Case
9c0275a879
Connection plugins network_cli and netconf (#32521)
* implements jsonrpc message passing for ansible-connection

* implements more generic mechanism for persistent connections
* starts persistent connection in task_executor if enabled and supported
* supports using network_cli as top level connection plugin
* enhances logging for persistent connection to stdout

* Update action plugins

* Fix Python3 RPC

* Fix Junos bytes<-->str issues

* supports using netconf as top level connection plugin

* Error message when running netconf on an unsupported platform
* Update tests

* Fix `authorize: yes` for `connection: local`

* Handle potentially JSON data in terminal

* Add clarifying detail if possible on ConnectionError
2017-11-09 15:04:40 -05:00
Brian Coca
ee80215fa9
Combine lists of dicts (#32686)
* allow combine to take lists of dicts

* updated as per feedback
2017-11-09 10:02:38 -05:00
Brian Coca
db749de5b8 namespace facts
updated action plugins to use new guranteed facts
updated tests to new data clean
added cases for ansible_local and some docstrings
2017-11-09 09:48:14 -05:00
Anil Kumar Muraleedharan
9d98452032 New enos_facts, + module_utils/enos.py. modifying copyright year in rest all (#31696)
* Squashing all commits to one as suggested by John

* Adding Unit test method for the module enos_facts.py

* Pep8 and Ylint issues addressed

* Trying again to remove blank line. Some scripts are required for this.

* Bug Fixing for interfaces

* Editing for over indenting issue

* E203 whitespace before ','

* Update enos.py

Added warnings argument as to check_args method

* Update enos_facts.py

Added warnings to check_args method
2017-11-08 15:55:46 +00:00
Jan Pazdziora
e5dbf63b65 Fix #31694: running with closed stdin on python 3 (#31695) 2017-11-08 11:59:42 +01:00
Denver Janke
57106662db Get (potentially) new composite host vars from inventory (#32606)
* Get new composite host vars from inventory

* Remove spaces from blank line
2017-11-07 23:10:00 -05:00
David M. Lee
1fd454e5b5 Add yaml output plugin (#32246)
* Add yaml output plugin

Using YAML instead of JSON for detailed output can (IMO) improve
readability; especially for tasks with either lots of output, or
multi-line output.

* Fix yaml output plugin documentation

 * Added copyright header to yaml.py
 * Moved mention of yaml output plugin to `New Modules` section

* Use AnsibleDumper in yaml output plugin

Also moved the `represent_scalar` setup into the init method. As a
global statement, it was causing exceptions trying to get an `id`
field that does not exist.

* Remove trailing whitespace

* Remove useless json.loads/dumps from yaml output
2017-11-07 22:00:11 -05:00
paulquack
a5da2e44a1 ironware_config module (#32187) 2017-11-07 08:22:22 +00:00
Kenneth D. Evensen
4eccb447cf Lookup plugin for the OpenShift Container Platform. 2017-11-06 19:21:52 -05:00
Matt Davis
8ecc7bc4a1
minor become/runas cleanup (#32564)
* removed/blobified unused PInvoke stuff
* added try/finally around impersonation to ensure RevertToSelf is called in all cases
* added a few explanatory comments
2017-11-06 11:24:04 -08:00
Jordan Borean
15b492ca57 win_become: get admin token and fix async (#32485)
* win_become: make it easier to become with an admin token

* Fixed up pep8 whitespace

* fix for Server 2008

* Added support for async and become on newer hosts and fix warnings
2017-11-03 16:14:48 -07:00
Sam Doran
e65045f51f Add echo option to pause module (#32205)
* Enable ECHO in prompt module

Fixes #14160

* Add option for controlling echo behavior with pause module

* Improve option logic

Allow all options to be used in varying combinations, rather than being mutually exclusive.
Always capture output and return it, even when a time limit is set.

* Add version_added to docs

* Improve behavior of echo output

Set a few more flags to allow interactive deletion and hide control characters.
Do not capture or echo input when a time is set. Tried to get this working nicely, but ran into too many issues/oddities to keep it. Maybe in the future if there is demand for capturing/echoing input when a time is set I'll take another pass at it.
2017-11-03 13:32:36 -04:00
Brian Coca
e582a69e9c
show specific undefined var errors on -v in debug (#32206)
* show specific undefined var errors on -v in debug

allows users to get more specific information about undefined errors
as they might be looking at a complex data structure and need to find
the specific leaf that has the issue

* now tests works at all verbosity levels

* updated to unicode objects
2017-11-02 18:11:31 -04:00
Brian Coca
9d28973b5e ini plugin should recursively instantiate pending
solves inconsistent behaviour on ini host format depending on definition order
fixes #32196
2017-11-02 17:57:20 -04:00
Brian Coca
0f7fcd2b40
config lookup plugin (#32392)
* config lookup plugin

* added info about ansible-config list

* fixed typoes

* now with option and error handling

* fixed import, better docs
2017-11-01 16:38:03 -04:00
Brian Coca
4fb9e54c50 calculate max fail against all hosts in batch
currently it is doing only from the 'active' hosts in the batch which means
the percentage goes up as hosts fail instead of staying the same.
added debug info for max fail

fixes #32255
2017-11-01 08:45:31 -07:00
Brian Coca
41685fb516 fix precedence issue with facts and vars (#32302)
avoid making gathered facts high precedence, only set_fact is supposed to be.
vars set via set_fact with cacheable are higher precedence than plain facts.

Previously (after 6fbd0a8bb5) regular facts would end up with a
higher precedence than host or play vars, but they should not be. Facts were getting added to 'non_persistent_facts' (equivalent to 'register' vars) which is higher precedence than facts should be.

added 'cacheable set_facts' to precedence docs

'ansible_facts_cacheable' ->  '_ansible_facts_cacheable' (made 'private')
2017-11-01 11:42:17 -04:00
Ganesh Nalawade
26583adb58
Fix wrong prompt issue for network modules (#32426)
* Fix wrong prompt issue for network moodules

Fixes #31161
Fixes #32416

*  Store the device prompt in case of error
   from remote device
*  Check for prompt value in ios action plugin

* Add integration test
2017-11-01 19:38:19 +05:30
Brian Coca
e70c0afc5e now with_dict allows for direct 'lookup' usage
fixes #32067
2017-10-30 14:52:58 -04:00
James Tanner
ded0b6ed83 Change the ssh+sudo debug messages to vvv for easier troubleshooting in awx+tower 2017-10-30 10:41:50 -04:00
Patrick Ogenstad
81f34cd01c Remove asa_template action plugin (#32233)
Since the asa_template module has been deprecated this action plugin
is no longer needed.
2017-10-27 13:53:09 +01:00
Stanley Karunditu
7553c42e09 Fixes #31090. In network parse_cli filter plugin, this change moves the creation of a (#31092)
new match(block). It previously only occurred if a new match occurred,
but suggest it should occur when an end of a block match is found.
2017-10-27 17:45:36 +05:30
Sam Doran
6a6ea663ea Return all elements in a more robust way
If a trailing ':' is set or not, always return all secrets from a path. This was a bug mostly fixed by this PR.

Update examples.
2017-10-26 13:24:39 -04:00
igor-pinchuk
6b6746dcee Update hashi_vault.py
Add ability to lookup entire objects in HashiCorp Vault.
When used with auth_method allow skipping SSL verify.
2017-10-26 13:24:39 -04:00
Sam Doran
104934c095 Enable ECHO in prompt module (#32083)
* Enable ECHO in prompt module

Fixes #14160

* Set flags to make it possible to edit echoed input as well as hide control charcters

Only do this if a time limit is not set.

* Consolidate settings
2017-10-26 12:05:08 -04:00
James Mighion
11153df0a1 Adding a regex for stderr catching. (#31945) 2017-10-25 14:51:49 -07:00
John R Barker
0e36854213 consul_kv: Fix name & formatting (#32132)
* Fix name & formatting
* Fix examples
2017-10-25 15:39:49 +02:00
akatch
ad3f4b953a Fun with callbacks (#32031)
* Functional-ish unix-ish callback plugin

* fix var

* Don't display include or include_vars

* Correctly display debug output and failed output in loops

* Linter fixes, documentation, and GPL oneliner
2017-10-24 12:38:49 -04:00
Peter Sprygada
905d71d46a fix eos terminal plugin to recognize ospf error message (#32039)
The eos terminal plugin did not correctly catch the error message
returned with trying to configure more than one ospf instance.  This
change updates the terminal plugin to catch that scenario
2017-10-23 17:27:58 -04:00
Jordan Borean
b004a6373a windows: add #AnsibleRequires for Windows modules (#31683)
* windows: add #AnsibleRequires to set whether a module requires module or a specific version

* fix up pep8 issues

* changed psversion to use the actual ps Requires -Version syntax

* missed the check on #Requires -Version

* fix #Requires module extensions
2017-10-23 14:21:30 -07:00
Martin Krizek
4bd16216fe ini inventory: document value parsing workaround
Fixes #31656
2017-10-23 09:56:49 -07:00
Charles Tolley
531d6c50bf Added regex to recognize prompt (#31947) 2017-10-23 19:34:10 +05:30
James Mighion
6693cc3f5a Using shorthand for disabling paging to avoid the syntax difference between versions. (#31942)
Fixes #31855
2017-10-23 10:15:30 +02:00
Brian Coca
fd574c069e remove fixme from docs 2017-10-20 09:59:23 -04:00
James Mighion
bb4b98a82e Adds regex for stdout needed for changing root password and regex for stderr catching problems with the password. (#31955) 2017-10-19 16:52:14 -07:00
Brian Coca
2ffe3c42bb fixed caches (#30667) 2017-10-19 14:23:22 -04:00
Jerry Chong
567e989581 Fix exception upon display.warn() (#31876)
Fixes #31875
2017-10-18 13:04:59 +02:00
paulquack
5a6ee054c0 Network command module for Brocade IronWare routers (#31429) 2017-10-17 13:54:32 +01:00
Brian Coca
21cdddce74 correctly deal with changed (#31812) 2017-10-17 11:44:17 +10:00
Chris Meyers
cf938e9992 tests for InventoryModule error conditions (#31381)
* tests for InventoryModule error conditions

* modified unicode in tests to ahear to Ansible best practices

* flake8 fixes
2017-10-16 18:52:44 -04:00
Brian Coca
01b6c7c9c6 better cleanup on task results display (#27175)
* better cleanup on task results display

callbacks get 'clean' copy of result objects
moved cleanup into result object itself
removed now redundant callback cleanup
moved no_log tests

* moved import as per feedback
2017-10-16 09:44:11 -04:00
Marat Bakeev
f84b5b6059 Update ssm lookup to support getting parameters by path (#31752) 2017-10-16 10:31:57 +10:00
Ganesh Nalawade
89428a40b3 Validate parse_cli filter inputs (#31293)
* Validate parse_cli filter inputs

Fixes #30517

*  Add check to validate input is of type string
*  Add check to confirm template file exist

* Update error message for parse_cli_textfsm invalid template path

* Add input validation for parse_cli_textfsm filter
2017-10-13 21:26:24 +05:30
Ryan Brown
e0c2b6bbb4 Rename ssm_parameter_store module and ssm lookup to be prefixed with aws (#31662) 2017-10-13 13:52:10 +10:00
James Mighion
50052b3d70 Adding a cli transport option for the bigip_command module. (#30391)
* Adding a cli transport option for the bigip_command module.

* Fixing keyerror when using other f5 modules. Adding version_added for new option in bigip_command.

* Removing local connection check because the F5 tasks can be delegated to any host that has the libraries for REST.

* Using the network_common load_provider.

* Adding unit test to cover cli transport and updating previous unit test to ensure cli was not called.
2017-10-12 15:07:15 -07:00
Brian Coca
1bf09a7d84 warn on bad keys in group 2017-10-12 12:02:11 -04:00
Mihai Moldovanu
79f2cfbcf9 etcd.py module improved . added posibility to configure from playbook… (#18580)
* etcd.py module improved . added posibility to configure from playbook etcd server to query

* Update playbooks_lookups.rst

documented etcd lookups

* Update playbooks_lookups.rst

* Update playbooks_lookups.rst

* Update playbooks_lookups.rst

* Update playbooks_lookups.rst

* Update playbooks_lookups.rst

* Fixed errors reported by ansibot

* Update playbooks_lookups.rst

* Update playbooks_lookups.rst

* Copy edits.
2017-10-12 15:55:59 +01:00
Brian Coca
b68fe2cd23 highlight dynamic parts of error msg 2017-10-12 10:00:06 -04:00
Anil Kumar Muraleedharan
ead00e9cae create terminal plugin for enos modules (#31510) 2017-10-12 13:32:27 +01:00
Anil Kumar Muraleedharan
fa9421ef98 create cliconf plugin for enos - enos.py (#31509) 2017-10-12 13:31:57 +01:00
Anil Kumar Muraleedharan
34cc563727 Create action file enos.py (#31508)
* Create action file enos_facts.py

* Update and rename enos_facts.py to enos.py

* Taking chances on Dealing with Unstable issues

* Removing blank space/ white line
2017-10-12 13:31:23 +01:00
Jordan Borean
7fce636c2b windows: removed #Requires from PS pipelines as it causes STDERR false positives (#31558) 2017-10-11 14:16:58 -07:00
Brian Coca
2ed46e04f4 more updates to plugin/config generation (#30787)
* fixed module generation

added missing lookup page
point to plugins when plugins
made modules singular
add display for verbose an debug messages
nicer templating, changed generation order for ref
corrected links
moved most of lookup docs to plugin section

* Copy edits
* Fixed typos
* Clarified wording
2017-10-11 00:15:25 -04:00
Matt Davis
a4ceb4c35f improve become/runas error messaging on bogus/missing username (#31551) 2017-10-10 16:53:21 -07:00
François Scala
aade5234a9 Fix urlparse import for Python3 (#31240)
* Fix urlparse import for Python3 in

* contrib/inventory/consul_io.py
* contrib/inventory/rudder.py
* contrib/inventory/windows_azure.py
* lib/ansible/module_utils/known_hosts.py
* lib/ansible/modules/cloud/centurylink/clc_firewall_policy.py
* lib/ansible/modules/cloud/docker/_docker.py
* lib/ansible/modules/cloud/ovirt/ovirt_disk.py
* lib/ansible/plugins/action/ce_template.py
2017-10-09 20:18:09 -07:00
Jordan Borean
82fa922491 win_become: Added support to become a service user (#30407)
* win_become: Added support to become a service user

* fixes for linting

* changes to get local and network service working

* fixed linting issues again

* pleasing pepe
2017-10-09 17:58:29 -07:00
Brian Coca
4e22677e7d revert namespaced facts changes to package/service
fixes #31485
2017-10-09 16:31:58 -07:00
Bill Wang
91a36faddb New module: Add module for Amazon Systems Manager Parameter Store (cloud/amazon/ssm_parameter_store) (#23460)
- new module: ssm_parameter_store
- new lookup: ssm

* lookup module ssm - adjust error message

* Pacify pylint erroring on botocore not found

* adjust to version 2.5
2017-10-10 09:04:40 +10:00
Jordan Borean
f2bc89a880 win_copy: added decrypt option (#31291) 2017-10-10 08:21:25 +10:00
Brian Coca
e4c61ea9a1 fix 'return false' from parse
this was abandoned early on the manger side  but seems like we left behind on plugin side.
more flexible extensions with yaml plugin
validate data correctly for yaml/constructed
fixed issue with only adding one child to keyed, the group only got the host that forced it's creation

fixes #31382
fixes #31365
2017-10-06 13:25:06 -04:00
Brian Coca
f2ade09dce use configured ansible_shell_executable
refine args/shell/executable hanlding
2017-10-06 08:56:21 -04:00
Jordan Borean
c4b51009f0 win_copy: allow src to have invalid windows chars (#31392)
* win_copy: allow src to have invalid windows chars

* create file with colon instead of avoid checkout errors
2017-10-06 15:23:37 +10:00
Ganesh Nalawade
2e4dcc9dda Fix nxos terminal regex to parse xml response (#31283)
Fixes #26130

Modify nxos terminal regex to parse a xml response.
2017-10-05 15:32:43 +05:30
Brian Coca
8aa33419c9 check type to avoid typeerror tb
fixes #31290 by giving more meaningful message
2017-10-04 15:14:10 -04:00
Brian Coca
46621a3ddd updated constructed docs to clarify use of ext 2017-10-04 08:44:45 -04:00
Jordan Borean
e61c2799ff win_become: move error handling to Ansible outside of shell (#31227)
* win_become: move error handling to Ansible outside of shell

* trimmed the output so double newlines don't get set

* added test for non-zero exit code

* missed issue URL on test

* changed exit to SetShouldExit
2017-10-03 21:42:27 -04:00
Jordan Borean
bba941cd5b win_copy: fix for copying encrypted file without pass (#31084)
* win_copy: fix for copying encrypted file without pass

* fix pep8 issue

* reduced the diff and fixed some minor issues
2017-10-03 12:51:57 -07:00
Adrian Likins
dae0ad1ce6 Fix flush_cache on redis cache
Fix a del on non existent self.cache.

Fixes #31148
2017-10-03 12:54:26 -04:00
Brian Coca
101377768b remove unused ssh pipelining setters
fixes #31125
2017-10-03 09:22:01 -04:00
Abhishek Shukla
d0c003ab0f BugFix code breaks, if include_vars dir is file instead of directory (#31157) 2017-10-03 00:00:51 -04:00
Julien Palard
5802ec04bf yaml inventory: Better error reporting on typo. fixes (#31118) 2017-10-02 09:22:21 -04:00
Brian Coca
ac9278ff0f remove action plugin only fields from 'file' calls (#31047)
* remove action plugin only fields from 'file' calls

fixes #30556

* Add a test for #30556
2017-09-29 17:13:32 -07:00
Brian Coca
43cbcbcc75 removed typo from dig docs 2017-09-29 11:23:58 -04:00
Brian Coca
057eec94ee fixed winrm to use proper task vars (#31072)
it avoids hitting hostvars templating issue and ignoring exception
fixes #30911

also normal var precedence should work for ansible_winrm vars
2017-09-29 11:12:56 +10:00
Ganesh Nalawade
97858d91ac Add alias to command option in telnet (#30825)
*  Add commands alias to telnet option
2017-09-28 18:07:37 +05:30
James Mighion
2837200aea Adding new stdout regex to allow local password changes. Adding stderr regex to catch more errors. (#30994) 2017-09-28 10:05:14 +05:30
Brian Coca
54d7c384b6 clarified dig module uses dnspython 2017-09-27 20:59:03 -04:00
Brian Coca
bb874e5029 fixed lookup dig docs 2017-09-27 20:05:21 -04:00
Brian Coca
82de2e1865 rmeove ext requirement for yaml inventory plugin
returns to the state before 2.4 made it a requirement
fixes #30855
2017-09-27 14:28:07 -04:00
Brian Coca
8378ac30b3 fix for slack callback breaks in 2.4 (#30932)
* fix for slack callback breaks in 2.4

fixes #30838
2017-09-26 17:18:59 -04:00
Peter Sprygada
909100bd2c fixes issue where filter wouldn't error on undefined var (#30921)
The filter will now correctly error on an undefined variable when trying
to template the key `value`
2017-09-26 11:50:57 -07:00
Ivan Pepelnjak
9aa1da23f0 Add one-versus-many-matches logic to network filter plugin (#30511)
* Add one-versus-many-matches logic

* Fixed indentation (tabs ==> spaces)
2017-09-26 05:44:07 -07:00
Brian Coca
b233f3f296 updated plugin docs (#30490)
* updated  docs

- for devs:
  - added inventory/vars section
  - made some updates to general section and other plugin types
- for users:
 - added 'user' plugin section to start describing the plugins
 - docs on types, what they are and how to use

- removed ref to deleted AUTHORS file
- corrected several typos/headers
- added descriptions to config.rst template
- ignore generated files for cli/plugins and config
- remove new generated files on `make clean`
- moved details from devguid and intro doc to plugin specific pages
- pretied up lookup notes
- changed precedence ref to not conflict config
- removed duplicate config data, as config is autogenerated and up to date
- put new plugins under playbooks
- added `pass` cause rst/python dislikes fractions
- removed dupe in .gitignore, alpha sorted to avoid moar dupes
- added try cause rst/python freaks out

* generate plugins into their own dir

only do plugins that support docs
use toctree from main plugins page
2017-09-22 23:19:50 -04:00
Brian Coca
a819cfcad7 dont validate group names in yaml plugin 2017-09-22 19:05:32 -04:00
Ganesh Nalawade
d72eb08902 Fix nxos provider transport warning issue (#30610)
* Fix nxos provider transport warning issue

*  Add default value of transport arg in provider spec
*  Remove default value if transport arg in top level spec
   This ensure deprecation warning is seen only in case transport
   is given as a top level arg in task
*  Refactor nxos modules to reference transport value from provider
   spec

* Fix unit test

* Remove transport arg assignment in nxos action plugin

* As assigning transport value is handled in provider spec
  top level task arg assignment is no longer required
2017-09-22 10:32:25 +05:30
Dag Wieers
2392934f48 More PEP8 fixes
Not sure why it missed those.
2017-09-21 05:16:49 +02:00
Dag Wieers
63c6afeaee Fix PEP8 issue 2017-09-21 05:07:49 +02:00
Dag Wieers
57b85198fe Add support for loop-tasks and improved output
This PR includes:
- Support for loop-tasks with proper subject/error content
- Improved output (and proper indentation)
- Complex data structures are now pretty printed
- Better selection of mail subject
2017-09-21 04:54:26 +02:00
Toshio Kuratomi
72bdf7aa7e Fix python3 and nonascii handling in inventory plugins
Fixes #30663
2017-09-20 19:39:15 -07:00
Brian Coca
9f4d73b699 fixed credstash docs 2017-09-20 20:52:52 -04:00
Patrick Ogenstad
d74c871559 Restore correct coloring to selective callback (#30528)
* Restore correct coloring to selective callback

This fixes the bug raised in #30506

* Fix format issues for Python 2.6 & indent

Removed the zero length fields to support format under Python 2.6

Fixed E128 continuation line under-indented for visual indent issue
2017-09-20 09:14:59 -04:00
Brian Coca
71cc906ec8 fixed password lookup docs 2017-09-19 16:33:15 -04:00
Brian Coca
b9120dfd97 fixed decreption
(cherry picked from commit 1cee81375386a7f3ac3d1893342b9323983f00c9)
2017-09-19 11:43:36 -04:00
Brian Coca
24d4787b2d Lookup docs (#30280)
* finalize lookup documentation
* minor fixes to ansible-doc
 - actually show which file caused error on when listing plugins
 - removed redundant display of type and name
* smart quote fixes from toshio
2017-09-19 10:49:07 -04:00
Matt Clay
7714dcd04e Enable more pylint rules and fix reported issues. (#30539)
* Enable pylint unreachable test.
* Enable pylint suppressed-message test.
* Enable pylint redundant-unittest-assert test.
* Enable pylint bad-open-mode test.
* Enable pylint signature-differs test.
* Enable pylint unnecessary-pass test.
* Enable pylint unnecessary-lambda test.
* Enable pylint raising-bad-type test.
* Enable pylint logging-not-lazy test.
* Enable pylint logging-format-interpolation test.
* Enable pylint useless-else-on-loop test.
2017-09-18 23:20:32 -07:00
hiddenicon
33b8d7069f Add _ and . to regex (#30396)
Adding underscore and period to the nxos regex for determining the prompt for hostnames with underscores and periods in the hostname.
2017-09-18 14:59:11 +05:30
Brian Coca
ae29245e05 decelerate! (#30160)
removed accelerate code
removed keyczar dep for accelerate
2017-09-16 23:35:50 -04:00
Brian Coca
fc9b44e021 clear correct cache when reconcile is invoked
moved it to ensure all callers get the correct hosts after
2017-09-15 12:58:40 -07:00
Brian Coca
81fd67c10f fixed issue with default callback inheritance (#30427)
* fixed issue with default callback inheritance

 - callbacks need to document same options as callbacks they inherit from to get them configured
 - since default is also used by many 3rd party callbacks for inheritance, making the code 'tolerate' the missing docs
   and fallback to using the direct constant to configure it's options.
2017-09-15 11:14:47 -07:00
Nathaniel Case
5d404af76e Raise an error if ansible_network_os is not specified (#30429) 2017-09-15 13:56:05 -04:00
James Cammarata
6730f81024 Correctly create include_role blocks when using loops (#30372)
Also fixes instances where conditionals or other variables may result in
hosts executing lists of tasks of differing sizes.

Fixes #18748
2017-09-14 15:56:12 -05:00
Ganesh B. Nalawade
8c3bf20a13 Use safe_load to load yaml
Replad `yaml.load` with `yaml.safe_load`
2017-09-14 11:19:00 -07:00
Patrick Ogenstad
58088e836e Fix import error. Was pointing to the wrong module (#30293) 2017-09-14 10:24:28 -07:00
Mark Goddard
ac69fcccdc Fix Dell OS network module timeout (#30355)
The dellos action plugins should add the remote address of the switch
provider to the play context. This was fixed in issue #23589 in an
almost identical manner for the eos, ios, iosxr, and vyos action
plugins.

Fixes: #30350
2017-09-14 15:40:50 +01:00
Marek Hulan
9ad90de4bc Fix sending facts to Foreman 2017-09-13 13:58:45 -07:00
Trishna Guha
578ae3b238 fix nxos_file_copy and provider getting set to None when transport is cli (#30262)
* nxos_file_copy bug fix

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* provider gets set to None in module level when transport is cli

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
2017-09-13 16:36:41 +05:30
follower
8489e298ca doc: Typo fix: "accesability" --> "accessibility" (#30251) 2017-09-13 11:17:29 +02:00
Matt Clay
442af3744e Miscellaneous pylint fixes.
The following rules are no longer disabled:

- bad-format-string
- duplicate-key
- lost-exception
- trailing-newlines
- unexpected-keyword-arg
- useless-suppression
- using-constant-test
2017-09-13 01:53:08 -07:00
Jordan Borean
77b2aca5a2 fixed become to show the stdout and stderr streams instead of the StreamReader (#30254) 2017-09-13 17:35:12 +10:00
abirami-n
adc533804a module_utils_fix_dellos6 (#28060) 2017-09-12 18:24:36 +01:00
Ganesh Nalawade
530988666a Allow delegate_to if transport is not cli (#29945)
Fixes #29060

Allow delegate_to if transport is either nxapi or eapi.

Persistent connection uses `cli` transport and create
a local socket on control node. Hence delegate_to is not allowed
for `cli` transport.

However as `nxapi` and `eapi` transport does not use persistent connection
it is possible to use delegate_to in this case.
2017-09-12 21:34:13 +05:30
Ganesh Nalawade
599fe23ed6 Fix unwanted deprecation message in network module args (#28984)
* Fix unwanted deprecation message in network module argspec

Fixes #25663
Fixes #24537

*  segregate provider spec and top level arg spec
*  add deprecation key in top level arg spec
*  remove action plugin code to load provider and add
   that logic at a common place in network_common.py file

* Fix CI issue

* Minor change
2017-09-12 13:00:01 +02:00
Matt Clay
68aeaa58a8 Fix dangerous default args. (#29839) 2017-09-12 00:11:13 -07:00
Toshio Kuratomi
cc343a4376 Port ansible doc for plugins to use DOCUMENTATION variables
* Using docstrings conflicts with the standard use of docstrings
* PYTHON_OPTIMIZE=2 will omit docstrings.  Using docstrings makes future
  changes to the plugin and module code subject to the requirement that we
  ensure it won't be run with optimization.
2017-09-11 18:34:09 -07:00
Brian Coca
2165bac212 module and vault fixes (#29663)
* module and vault fixes

- fix module_path cli option and usage, which fixes #29653
- move --output to be in subset of vault cli, no need for all vault enabled cli to use it
- added debug to loader to see directories added
2017-09-11 21:02:16 -04:00
Brian Coca
869a318492 allow config for callbaks and some fixes
* only complain about ini deprecation if value is set
 * set plugin config for stdout and other types
 * updated plugin docs, moved several plugins to new config
 * finished ssh docs
 * fixed some issues seen in plugins while modifying docs
 * placeholder for 'required'
 * callbacks must use _plugin_options as _options already in use
2017-09-11 14:00:01 -07:00
Toshio Kuratomi
ff13d58c14 Remove use of unicode_literals as it is an anti-pattern
from __future__ unicode_literals leads to developer confusion as
developers no longer can tell whether a bare literal string is a byte
string or a unicode string.  Explicit marking as u"" or b"" is the way
to solve the same problem in the Ansbile codebase.
2017-09-11 13:33:51 -07:00
Toshio Kuratomi
3277e6328b Define byte strings versus text strings in docker connection
Fixes python3 string handling in the docker connection plugin

Fixes #24776
2017-09-11 09:55:09 -07:00
Matt Martz
d043ba2673 Create a new pipe for sshpass on retries. Fixes #29095 2017-09-11 08:49:15 -07:00
Tomas Tomecek
0274835add conn,buildah: don't pass bytes to shlex.split (#27896)
* conn,buildah: don't pass bytes to shlex.split on python3
2017-09-09 17:40:07 -07:00
Toshio Kuratomi
75249e311e Fix set theory filters for python3
The set theory filters need to use lists rather than generators on python3.

Also add unit tests for most of the mathstuff filters

Fixes #26494
2017-09-09 17:38:39 -07:00
Pierre Guinoiseau
436b173b24 Fix py3 string issue in jail connection plugin (take 2) (#28374)
* Fix py3 string issue in jail connection plugin
2017-09-09 11:52:42 -07:00
Matt Clay
4de6076d78 Fix pylint issue. 2017-09-09 10:45:39 -07:00
Brian Coca
075ead8fb0 fixes to config/setting retrieval
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
2017-09-09 09:48:22 -07:00
Toshio Kuratomi
e1e2d08508 Combine exception handling to simplify code (#29096) 2017-09-07 13:20:44 -04:00
Brian Coca
13d1520f3d Db cache fix (#29048)
* cleaner get for file based caches
* now db based facts behave like file ones
we now keep local in mem cache to avoid race conditions on expiration during ansible runs
2017-09-07 12:17:16 -04:00
Brian Coca
0ddbcf8841 python3 fix 2017-09-06 09:42:58 -04:00
Ganesh Nalawade
cc9ed352dd Ensure proper conversion while backing up of junos config (#28958)
* Ensure proper conversion while backing up of junos config

* Minor changes

* Fix review comment

*  Open config backup file in binary mode
2017-09-04 20:12:11 +05:30
Sergey Scherbakov
c16d258a27 Fix encoding error on grp.gr_name, which can contain non-ascii chars at domain PCs (#25804)
* Fix encoding errors on grp.gr_name, which can contain non-ascii character at LDAP/AD domain workstations

* fix: utils.to_text() is now used instead of py3-incompatible unicode() method
2017-08-31 03:39:24 -04:00
Dag Wieers
47be50a450 mail callback: Much improved version (#28501)
So 5 years after the original version, it's time to give this callback
plugin some needed rework.

This PR includes:
- Reuse same logic for failed, unreachable or Async errors
- Add playbook and task information at the top (easier to find context)
- Improve the sections (ie. indented content)
2017-08-30 05:58:40 +02:00
Dag Wieers
513b582ba3 win_reboot: Use more conforming parameter names (#26466)
This is part of the effort to make windows parameters conform to
other modules. Usually parameters don't include the unit in the
parameter name.

See also #20160
2017-08-29 19:08:12 -07:00
Sam Doran
80c00d3238 Create urlsplit filter (#28537)
* Improve tests for uri filter

* Create URL Split docs

* Add urlsplit filter

* Py3 compatibility

* Use helper method and eliminate query options

* Add options, cleanup output, fix tests

* Update docs

* Add parenthesis to boilerplate import

* Add debug task to tests

* Use exclude option to filter returned values

* Filter out additional option for Python 3
2017-08-29 17:58:44 -04:00
Dag Wieers
eed6a724ea Bugfix to check-mode and async support, and features (#28448)
This PR includes:

- Check-mode support, the module will now report as if it was rebooted,
  without making the actual reboot.
- Async-support, this potentially fixes #23835
- The module now also returns the elapsed time, like the wait_for and wait_for_connection modules.
  This fixes #18108
- Add post_reboot_delay_sec parameter
2017-08-29 14:44:50 -07:00
Jacob McGill
6aaa0c3252 Net Ping: Add new action plugin for *_ping network modules (#28558) 2017-08-29 13:28:58 -04:00
Monty Taylor
d548c477c0 Translate openstack inventory from script to plugin 2017-08-29 12:45:41 -04:00
Brian Coca
c06f8a3f9b allow pesky 'bridge' facts to bypass facts filter (#28401)
* allow pesky 'bridge' facts to bypass facts filter

fixes #27729, #23577

* I ate pepe
2017-08-29 09:47:40 -04:00
Ganesh Nalawade
a914a39975 Convert value of port and timeout in provider to integer (#28732)
Fixes #28361
2017-08-29 18:40:10 +05:30
Brian Coca
95eaa246aa added keyed_group construction (#28578)
* added keyed_group construction

also added strict config to allow skipping bad templating
more precise error msgs
to_native better than to_text
fixed truthyness
added safe names

* allow keyed expressions to return lists

* PEPE should eat less, he is getting fat
2017-08-25 22:00:07 -04:00
Dag Wieers
1afbe29642 Remove transport_test for some connection types (#28450)
So we are removing the transport_test for the listed connection types,
because they fail to take into account bastion or proxy servers for
testing the transport.

The result of removing this, is that modules using this facility will do
a complete round-trip attempt, running a module, which is a bit heavier but correct.

This fixes #23774
2017-08-26 01:16:40 +02:00
Nicolas Porcel
ba420c47cd synchronize: Use private_key and ssh_user with delegate_to (Fixes #16215) (#21686) 2017-08-24 21:09:15 -04:00
Ganesh Nalawade
cd8c1c1108 Create persistent socket path using port and connection type (#28492)
* Create persistent socket path using port and connection type

*  Use remote address, port, connection type and remote user
   to create a socket path.

* Fix review comment
2017-08-23 18:28:44 -04:00
Jordan Borean
4382216e10 win_copy: changes that were missed (#28533) 2017-08-22 23:23:07 -04:00
Trishna Guha
c2e41bf1ef remove net_command (#28525)
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
2017-08-23 02:07:39 +05:30
Guido Günther
398f643c58 Check existence of SSL certs in Foreman callback (#25796)
* Fix typo

* foreman callback: check existence off SSL certs

This leads to confusing error messages later on otherwise.
2017-08-22 14:25:14 -04:00
Ganesh Nalawade
5d1d1a72c5 Fix junos integration test failure (#28491)
Pass ssh_config value to ncclient manager api as None
if it is not set.
2017-08-22 07:25:19 -04:00
Dag Wieers
96da0047ac Fix an issue causing win_copy to fail
It seems win_copy can fail because `will_change` was not returned in the dictionary when doing a query.
This is an easy fix to get it working again.
2017-08-22 13:03:32 +02:00
Brian Coca
a897193bce Moar constructive (#28254)
* made composite vars and groups generic

now you can do both in every plugin that chooses to suport it
renamed constructed_groups as it now also constructs vars ... to constructed
moved most of constructed_groups logic into base class to easily share

* documented inventory_hostname

* typo fix
2017-08-21 16:06:15 -04:00
Jordan Borean
8e40ac54dd win_copy: rewrite with new tests and functionality (#27678)
* win_copy rewrite with new tests and functionality

* minor pep fixes

* Handle UTF-8 filenames in zip

* fix for template

* when zip assemblies are not available in .net revert to old behaviour of copying one by one

* typo fix

* some more typos

* updated logic to correctly handle when new directories can be created

* removed testing file as it is not needed

* updated documentation based on PR
2017-08-18 20:33:37 -07:00
Peter Sprygada
03b888052b fixes bugs in network filter for parsing ip address (#28404)
also adds parse_cli and parse_cli_textfsm to changelog
2017-08-18 14:05:26 -04:00
Gaige B Paulsen
54c50cc983 netconfig module updated to pay attention to .ssh/config (#23876) 2017-08-18 12:04:30 -06:00
Bobby Watson
8e8a7c869a Changed string check to verify that EOS device is not in config mode. (#28332)
Changed string check to verify that EOS device is not in config mode. This was required in order to work with Arista 7500 series modular switches.
Resolves #2830
2017-08-18 09:06:46 -06:00
Mehran Kholdi
ecfa7f696d Enforce state='file' in copy module
This was causing wrong behaviour when `prev_state` was `hard`-link,
since the `file` module tried to apply the same `state` on the new
file, causing unexpected errors.

Particularly, both `overlay` and `devicemapper` storage drivers in
docker use hardlinks to share files between layers. This causes
most ansible playbooks to fail when working with files from layers
below.
2017-08-17 18:16:17 -07:00
Adrian Likins
01c0b2f714 Handle errors in jmespath in json_query better (#22109)
* Handle errors in jmespath in json_query better

Catch any exceptions raised from jmespath and raise
an AnsibleFilterError instead.

Avoid a traceback.

Fixes #20379

* pep8
2017-08-17 17:16:35 -04:00
Jonathan Davila
974d988703 Updated consul_kv lookup plugin so that it can take in host and port
parameters as keyword arguments instead of only being able to specify
it as environment variables.
2017-08-17 10:42:22 -04:00
Mike Wiebe
8f4bc8150f Fix default nxapi port setting when use_ssl set (#28236)
* Fix port setting when use_ssl set
2017-08-16 11:38:38 -04:00
Peter Sprygada
37524b798d updates network parse_cli filter to handle blocks from output (#28245)
* updates network parse_cli filter to handle blocks from output

* minor updates to finish up parse_cli filter

* all vars are now under the vars key
* attributes key has been changed to keys
* added the start_block and end_block directives

* update PEP8 failures
2017-08-16 08:59:14 -04:00
CyberArk BizDev
d50d65d448 Added cyberarkpassword lookup plugin (#21857)
* Added cyberarkpassword lookup plugin

Added cyberarkpassword lookup plugin: It allows to retrieve credentials
(password, sshkey) from CyberArk Digital Vault
2017-08-15 23:06:46 -07:00
Matt Davis
9b383403ce switch become/runas to LogonUser/CreateProcessWithTokenW (#28253)
* non-uac works

* switch become/runas to LogonUser/CreateProcessWithTokenW

* fixes #22218
* provides consistent behavior across authtypes
* auto-elevates on UAC if target user has SE_TCB_NAME ("Act as part of the operating system") privilege
* sets us up for much more granular capabilities later (eg, network/service/batch logons)
2017-08-15 18:55:17 -07:00
Brian Coca
f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
2017-08-15 16:38:59 -04:00
Sam Doran
a8e4c9be7a Cause copy module to fail on empty string as source (#27975)
* Fail if an empty string is set as src for copy module

Fixes #27363

* Cleanup task formatting on copy tests

Use multi-line YAML
Add debug statements with verbosity: 1 rather than leave them in there commented out.

* Add test for empty string as source

* Do more checks in order to add more specific errors messages

Add more integration tests for the various failure scenarios.
Cleanup some syntax on existing integration test tasks.
2017-08-15 12:41:16 -04:00
Nathaniel Case
20a35d0c1c Fix python3 str <-> bytes in connection/persistent.py (#28224) 2017-08-15 11:36:20 -04:00
Andreas Olsson
c9cdc3a259 Fix zip filter for Python3
Using the now bundled six library.

Fixes #28117
2017-08-14 23:03:15 -07:00
3onyc
554496c404 [passwordstore] Use builtin _random_password function instead of pwgen (#25843)
* [password] _random_password -> random_password and moved to util/encrypt.py
* [passwordstore] Use built-in random_password instead of pwgen utility
* [passwordstore] Add integration tests
2017-08-14 15:19:40 -07:00
Ganesh Nalawade
c9b2869f06 Add regex to retrieve socket path (#28181) 2017-08-15 00:17:20 +05:30
Toshio Kuratomi
3edac559d3 the smart transport is broken by ssh retry code
1fe67f9 introduced retries to the ssh connection put file and fetch
file.  Unfortunately, that change broke the smart transport because it
started raising exceptions instead of returning from _run().  This
breakage is documented in #23711.

An attempt to fix it was made at #23717 but the first attempt was
objected to as needing to touch too much code.  The second attmept was
objected to as smart was forced to encapsulate retries (thus retrying
a sftp "rety" times before trying scp "retry" times and then finally
moving onto piped).  This third attempt has retries encapsulate smart.
So each sub-transport is tried once and if all three fail, another retry
attempt is made which tries each of the three again.

Fixes #23711
Fixes #23717
2017-08-14 08:08:00 -07:00
Giovanni Sciortino
ab81c6c0f3 strategy: lost list order using with_items during displaying results Fixes #21008 2017-08-14 10:59:25 -04:00
Peter Sprygada
5ee7862793 update telnet module (#28100)
* accepts list of prompts to expect
* updates doc strings
2017-08-11 23:50:01 -04:00
Brian Coca
40d29936a4 telnet action plugin (#27983)
* draft telnet action plugin

docs in module

* updated per feedback

* fixed imports

* pep8 fixes

* added meta import even though its useless
2017-08-11 22:46:26 -04:00
Toshio Kuratomi
a3132e5dd6 Optimize template (#28044)
* Optimize template

* In fixing template to handle diff correctly #24477, I introduced more
  round trips to the remote end which slowed things down  The new code now
  uses one fewer round trips than the old code.
* Reimplement a large part of template by calling the copy action plugin
  instead of doing it in template's code.  This reduces the code in
  template and gives us one place to fix bugs and optimize.
* Add a follow parameter to template that mirrors the follow parameters
  for file and copy.
* Fix copy's diff handling (probably broken in my rewrite for in 2.4
  development)
* Adjusted when copy creates tmp dirs to rduce round trips in copy and
  template.

Fixes #27956
2017-08-11 17:50:49 -07:00
Jacob McGill
21dbde0abe fix issue with trailing white space when logging into asa (#28073) 2017-08-11 12:30:57 -04:00
Patrick Ogenstad
089226e372 Fixes for broken asa_config module (#27218)
* Fixes for broken asa_config module

* strip() prompt before checking
2017-08-11 11:42:42 -04:00
Brandon Schlueter
0e334ca821 Don't compare group_vars paths with bytestrings (#27922)
* Don't compare group_vars paths with bytestrings

* Compare spath with unicode string in VarsModule
2017-08-11 10:21:20 -04:00
Peter Sprygada
edc3507c98 restore the ability to issue commands over junos cli (#28025)
* restore the ability to issue commands over junos cli

* add warn message for invalid arguments

* fix issue when checking if provider transport is set
2017-08-10 17:57:32 -04:00
Rémi REY
7974ad34f1 add playbook runtime stat in the logstash callback stats (#25785) 2017-08-10 13:43:21 -04:00
Peter Sprygada
d8b005ec65 updates the prompt regexp for prompt handling in sros (#28015)
* thanks to @jurajama for tracking this down and providing the fix
* fixes #24192
2017-08-10 12:59:49 -04:00
Juan Manuel Parrilla
2cfb30fc91 Fixes #24414, Added support for a ETCD folders (#24432) 2017-08-10 07:44:05 -06:00
James Hogarth
7ec34b4cb2 ISSUE 25470 - update patch from review details 2017-08-09 13:38:16 -07:00
James
fc274bda8a ISSUE 25470 - junit report failure on changes
This commit provides an environment option to change the behaviour so
that it's possible to declare any changes shoudl be considered a junit
failure.

This is useful when carrying out idempotent testing to ensure that
multiple runs are safe and any changes should be considered a test
failure.

In a CI test of an ansible role the practice would be to run the role
once without this to configure the test system, and tehn to run a second
time including this environment vairable so that the CI engine
processing the junit report recognise any changes to be a test fail.
2017-08-09 13:38:16 -07:00
Eduardo Suarez-Santana
9339b0f4a7 Fix lookup into search paths when undefined.
A fileglob may issue a warning `Unable to find xxxx in expected paths` when `ansible_search_path` is not defined, because it loops over the characters in the string instead of looping over a list of one element.
2017-08-09 14:07:40 -04:00
flowerysong
e2b2ba6bc7 hashi_vault: fix token logic again (#27863)
The token is not mandatory, and there are more ways of setting it than
passing it in as a module arg.
2017-08-09 10:53:30 -04:00
Nathaniel Case
c1bf74283e Network load_config: Capture configuration output and display as warnings (#27851)
* Capture configuration output and display as warnings

* Don't break on nxapi

nxapi errors very loudly instead, so no need to muck about with warnings
2017-08-09 09:43:04 -04:00
Brian Coca
e4a997c97e removed debug print
fixes #27864
2017-08-07 20:44:35 -04:00
Juan Manuel Parrilla
5df4ff8983 Fixes #25057, Added Hiera Lookup (#25060) 2017-08-07 12:59:39 -06:00
Arthur Kepler
d828449611 Corrects default arg format used by datetime filter (#27612)
This ensures the default datetime format matches that of
datetime.datetime. Docs updated to match as well.
2017-08-07 11:33:25 -07:00
Ben Waters
be5e2251a7 Fixes #21536: Allowing the ability to pass aws session paramters to credstash lookup (#23077)
* Allowing the ability to pass aws session paramters to credstash lookup
Fixes #21536
https://github.com/ansible/ansible/issues/21536

* Default environment variable lookup

To get around non lookup of environment variables. First, check for environment variables, then look for parameters

* Following AWS Precedence

http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#config-settings-and-precedence
2017-08-07 11:33:59 -04:00
Jacky Hu
c770d657f3 Enable url lookup without a proxy
By introducing a use_proxy parameter to url lookup, the usage of proxy
is totally controllable.

(cherry picked from commit 11e4e5123532344740c61f1069b53387df290b9b)
from #23811
2017-08-04 14:20:43 -04:00
David Barroso
0f60041dea Handle better the many ways information can be passed around (#27506)
* Handle better the many ways information can be passed around

* Fixing pep8 complaints

* Removing message
2017-08-04 09:25:11 -07:00
Toshio Kuratomi
3f12fccd02 Fix several things causing tracebacks with unicode cwd (#27731)
Fixes #27511
2017-08-04 09:25:08 -04:00
Peter Sprygada
7b604368d3 adds new filter plugins for network use cases (#27695)
* adds new filter plugins for network use cases

* adds parse_cli filter
* adds parse_cli_textfsm filter
* adds Template class to network_common
* adds conditional function to network_common

* fix up PEP8 issues
2017-08-04 07:47:12 -04:00
Will Thames
11da85f938 timer callback plugin: handle timezone changes better
A playbook that does `timezone name=Australia/Brisbane` on
a host previously in UTC will appear to take 10 hours.

Improve the seconds handling for playbooks that take longer
than one hour.
Improve the hours handling for playbooks that take longer than
one day.

TZ change before:
```
Playbook run took 0 days, 10 hours, 0 minutes, 36055 seconds
```
After:
```
Playbook run took 0 days, 0 hours, 0 minutes, 55 seconds
```

Sleep for 100s more than one hour before:
```
Playbook run took 0 days, 1 hours, 1 minutes, 3641 seconds
```

After:
```
Playbook run took 0 days, 1 hours, 1 minutes, 41 seconds
```
2017-08-03 11:09:27 -04:00
Adrian Likins
6fbd0a8bb5 Add set_fact_persistent action and module. (#26153)
* Add 'cacheable' param to  set_fact action and module.

Used just like set_fact, except facts set with cacheable: true
will be stored in the fact cache if fact caching is enabled.

set_fact normally only sets facts in the non_persistent_fact_cache, so they
are lost between invocations.

* update set_facts docs

* use 'ansible_facts_cacheable' in module/actions result

* pop fact cacheable related items out of args/results

We dont want to use 'ansible_facts_cacheable' result item
or 'cacheable' arg as actual facts, so pop them out of the
dicts.
2017-08-02 15:57:58 -04:00
Abhijit Menon-Sen
20b0716948 Allow variables to be passed in to template lookup plugin (#18662)
This allows a single template to be evaluated with different values in
the same task. For example, with a template like 'x:{{a}}', one could do
something like this:

    - foo:
        a: "{{ lookup('template', 'x.j2', template_vars=dict(a=foo[item])) }}"
        b: "{{ lookup('template', 'x.j2', template_vars=dict(a=bar[item])) }}"
      with_items:
        - x
        - y

…and "a" and "b" would expand to different strings based on what we
passed in to the template lookup.
2017-08-02 09:29:27 +05:30
Matt Davis
f19ed387a7 fix non-pipelined output parsing in winrm.py (#27606)
* ensures stdout/stderr are converted to text type in py3 to avoid json serialization failures
2017-08-01 14:59:52 -07:00
Ganesh Nalawade
70ce394840 Persistent connection timer changes (#27272)
*  Add command_timeout timer that defines the amount
   of time to wait for a command or RPC call before
   timing out.
*  Remove connect_retries and connect_interval configuration
   varaible and replace it with connect_retry_timeout to control
   the timeout value of connection to local scoket.
*  Make required changes to netowrk action plugins and relevant
   network files in module_utils.
*  Required documentation changes.
2017-08-01 11:45:45 -06:00
James Mighion
4dd8f281d6 Adding aruba_config module (#27130)
* Adding aruba_config module.

* Fixing documentation.

* Forgot action plugin.
2017-08-01 11:44:26 -06:00
Ganesh Nalawade
448c461940 Convert port value to integer (#25939)
Fixes #25175

convert port value to integer explicitly
2017-08-01 11:27:04 -06:00
ikelos
03d3c6135d Convert the src entry to a templated entry. (#24026) 2017-08-01 11:19:08 -06:00
Ganesh Nalawade
bb998a3cd2 Add fix to read correct socket path recieved from ansible-connection (#27560)
Currently socket path is send from `ansible-connection` (running as background
process) over stdout. This can conflict with debug logs that are also send on
stdout resulting in incorrect socket path received by the main process.

To avoid this add a socket path delimiter string which is recevied by
main process and socket path is retrieved based on delimiter string.

This implementation will change in future when ansible-connection
framework is made more robust.
2017-08-01 19:32:25 +05:30
Nikita Chernyi
889495bf7f skippy callback plugin: hides banner for skipped tasks (#18621)
* Modified skippy callback plugin

* Added original skippy.py, moved plugin to full_skip.py

* Full skip: fixed code style
2017-08-01 08:19:51 -04:00
Jordan Borean
1517db06c5 fix to alllow the winrm plugin to send input with Python 3 (#27474) 2017-07-31 18:35:05 -07:00
Brian Coca
b79744f282 make random_choice more error resilient (#27380)
* make random_choise more error resilient

fixes #27373

* missing imports

* PEEP 16
2017-07-31 11:44:53 -04:00
Toshio Kuratomi
0a2cdb2585 New tests for copy recursive with absolute paths
Absolute path trailing slash handling in absolute directories

find_needle() isn't passing a trailing slash through verbatim.  Since
copy uses that to determine if it should copy a directory or just the
files inside of it, we have to detect that and restore it after calling
find_needle()

Fixes #27439
2017-07-28 21:00:51 -07:00
giovannisciortino
7cfd02097c add_hosts.py: Fix add_host does not recognise 'host' alias (#27418)
* add_hosts.py: Fix add_host does not recognise 'host' alias

* add_hosts.py: add note in documentation that 'hosts' alias of name parameter is only available on >=2.4
2017-07-28 17:35:20 -04:00
James Mighion
a328e96455 Adding aireos_config module (#27408)
* Adding aireos_config module.

* Fixing pep8 W291.
2017-07-29 00:36:04 +05:30
Toshio Kuratomi
cce06c5a3b Fix _get_file_contents to return bytes and checking that a different file exists than it opens 2017-07-28 11:07:43 -07:00
fredericve
095560771c Fixes #17958: use stderr instead of the default stdout for fatal errors (#17962)
* add a callback plugin that sends failures to stderr

* fix warnings
2017-07-28 10:50:15 -07:00
Martin Krizek
36c6d0f748 fetch: fail if flat=yes and dest=existing-dir w/o trailing slash 2017-07-28 09:53:50 -04:00
Matt Clay
51bd07204b Revert "Revert "Allow ini plugin to load file using other encoding than utf8." (#27407)"
This reverts commit 520696fb39.
2017-07-27 18:15:56 -07:00
Toshio Kuratomi
520696fb39 Revert "Allow ini plugin to load file using other encoding than utf8." (#27407)
* Revert "Update conventions in azure modules"

This reverts commit 30a688d8d3.

* Revert "Allow specific __future__ imports in modules"

This reverts commit 3a2670e0fd.

* Revert "Fix wildcard import in galaxy/token.py"

This reverts commit 6456891053.

* Revert "Fix one name in module error due to rewritten VariableManager"

This reverts commit 87a192fe66.

* Revert "Disable pylint check for names existing in modules for test data"

This reverts commit 6ac683ca19.

* Revert "Allow ini plugin to load file using other encoding than utf8."

This reverts commit 6a57ad34c0.
2017-07-27 17:08:31 -07:00
Yannig Perré
6a57ad34c0 Allow ini plugin to load file using other encoding than utf8.
- New option for ini plugins: encoding
  - Add a new option encoding to _get_file_contents
  - Use replace option in test/runner/lib/util.py when calling decode on stdout/err
    output when diff have non-utf8 sequences
2017-07-27 14:20:18 -07:00
Josh Bradley
31b27819e1 adding lookup plugin for chef_databags (#18403)
* adding lookup plugin for chef_databags

* add missing classes for code-smell
2017-07-27 16:42:32 -04:00
David Newswanger
81151ef02c Remove Deprecated Template network modules (#27076)
* removed deprecated networking template modules

* update changelog

* update changelog
2017-07-27 19:40:11 +01:00
mesk41in
0fb64214a4 add support of nested groups in group_by 2017-07-27 11:37:34 -04:00
Brian Coca
b93b9e68d7 ensure prefix in plugin loading to avoid conflicts
when using 'all' to load all plugins were ending in main namespace
creating conflicts with each other and random modulesr
 i.e. when trying to access json callback we were getting json 'parsing' lib
2017-07-26 11:54:13 -04:00
Toshio Kuratomi
225fa5d092 Fix undefined variables, basestring usage, and some associated python3 issues 2017-07-25 15:58:23 -07:00
Brian Coca
96d577c4c9 endswith supports tuples, nicer 2017-07-24 16:49:01 -04:00
Brian Coca
c7e841e0e4 better reading of 'facts' 2017-07-24 16:35:24 -04:00
Toshio Kuratomi
fc5d71de0d Fix for to_text and to_bytes error handlers
* surrogate_then_strict doesn't exist.  Switch to surrogate_or_strict
  instead.
* Found some bugs in the _text.py implementation
  * The composed error handlers (error handlers which are made up of two
    or more python encoding error handlers) had a wrong string in it,
    'surrogate_or_escape' doesn't exist.  Replaced that with
    'surrogate_or_replace' which is the correct handler name.
  * Left comment about the implicit conditions that are part of the
    surrogate_then_replace code path

Fixes #23865
Fixes #23861
2017-07-24 09:19:04 -07:00
Toshio Kuratomi
007a3b9c9b Don't need to convert cmd to bytes as it is already an array of bytes
Fixes #27016
2017-07-21 11:55:43 -04:00
Brian Coca
aca11a28a2 allow for null values in yaml inventory (#26765)
* allow for null values in yaml inventory

* 3 dad3 dad3 dad3 dad3 dad3 dad3 dad3 dad
2017-07-20 20:28:09 -04:00
Ricardo Carrillo Cruz
857726e6b1 Make the 'not found' regex case sensitive (#27100)
Turns out in some commands, like 'sh bgp summary' a line containing
'Not found' can be shown as a column.
This causes those commands to error out with RC=1 .

Fixes #26577
2017-07-20 12:33:18 -07:00
Gabor Lekeny
7eab802669 SSH fails with '"parsed": false' error message
Fixes: #15436
2017-07-20 08:09:42 -07:00
Toshio Kuratomi
f86ce0975d Add a directory walker to copy
* We need a directory walker that can handle symlinks, empty directories,
  and some other odd needs.  This commit contains a directory walker that
  can do all that.  The walker returns information about the files in the
  directories that we can then use to implement different strategies for
  copying the files to the remote machines.
* Add local_follow parameter to copy that follows local symlinks (follow
  is for remote symlinks)
* Refactor the copying of files out of run into its own method
* Add new integration tests for copy

Fixes #24949
Fixes #21513
2017-07-20 08:01:29 -07:00
Toshio Kuratomi
753a3a03d0 Revert "Fix for recursive copy slowness"
This reverts commit 78ced5318f.

The fix for copy slowness did not handle circular symlinks.
2017-07-20 08:01:29 -07:00
David Newswanger
0b5b5ec50a fix unicode errors in iosxr 2017-07-19 14:51:35 -07:00
Will Thames
ef8c9798d3 include_role handlers bug fix (#26335)
* Ensure that include_role properly fires handlers

include_role needs to ensure that any handlers included
with the role are added to the _notified_handler and
_listening_handler lists of the TaskQueueManager, otherwise
it fails when trying to run the handler.

Additionally, the handler needs to be added to the
PlayIterator's `_uuid_cache` or it fails after running
the handler

Add more uuid debug statements - this code was hard
to debug with existing debug statements, so add more
uuid information at little additional output cost.

Fixes #18411

* Add tests for include_role handlers

Tests for #18411
2017-07-19 15:02:32 -05:00
Michael Scherer
1289322147 Initial working prototype of a salt bus connection plugin (#12836) 2017-07-19 12:48:50 -06:00
Nathaniel Case
56a0b988a9 nxos integration fix part 1 (#27069)
* Assorted Python 3 fixes

* Fix `testcase` definition in integration tests

* Fix nxos_acl_interface

* clean up nxapi after nxos_nxapi
2017-07-19 14:00:05 -04:00
James Mighion
b8337ee9d3 New module aireos_command (#26769)
* Adding ciscowlc_command module and unit tests.

* Adding __init__.py for unit test.

* Fixing PEP8 W503.

* Renaming module from ciscowlc_command to aire_command.

* Renaming aire_command to aireos_command.
2017-07-19 23:14:52 +05:30
Andreas Olsson
779d05aec4 Support tilde (~) paths for unarchive creates tests
This expands the value of the creates parameter the same way which the
value of the dest parameter is already being expanded.

Fixes #26965
2017-07-19 11:13:17 -04:00
James Mighion
f682d9bf49 Adding aruba_command module along with unit tests. (#26625)
* Adding aruba_command module along with unit tests.

* Fixing PEP8 E303 too many blank lines.

* Adding default for timeout.

* Removing unused arguments. Moving default for timeout argument. Fixing cliconf to find hostname.

* Fixing PEP8 E302.
2017-07-19 09:49:12 +05:30
Matt Davis
907b662dc6 Powershell module_utils loader and tests (#26932)
* supports custom module_utils loads (anything in module prefaced with `#Requires -Module Ansible.ModuleUtils.*`)
* supports all usual PluginLoader module_utils locations (built-in lib/ansible/module_utils/, custom path from config, playbook module_utils/, ~/.ansible/module_utils, role module_utils, etc), 
* moves Powershell module_utils from module_utils/powershell.ps1 to module_utils/powershell/Ansible.ModuleUtils.PowerShellLegacy.psm1
2017-07-18 20:44:01 -07:00
David Moreau-Simard
26d358c64d Fix typo in docstring for host inventory plugin 2017-07-18 18:41:52 -04:00
Matt Clay
6ce683c29e PEP 8 fix. 2017-07-18 10:22:34 -07:00
Brian Coca
5ffb40fcdb properly filter out noise from debug 2017-07-18 12:32:39 -04:00
Peter Sprygada
8e2dcaf9f6 update asa to use network_cli connection plugin (#26899)
* WIP update asa to use network_cli connection plugin

* add asa.py to cliconf plugins
* update asa.py terminal plugin to support regexp and events
* update constants to map asa modules to asa action handler
* update asa action handler to implement persistent connections
* update asa shared module to use persistent connections
* update asa_command module to use new connection

* fixed pep8 issues
2017-07-17 21:23:38 -04:00
Toshio Kuratomi
ff22528b07 Consolidate boolean/mk_boolean conversion functions into a single location
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.

Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
2017-07-17 11:48:05 -07:00
Brian Coca
04868c22b7 fixed issue with null host vars 2017-07-15 00:20:08 -04:00
Toshio Kuratomi
e986e31a75 This function is for setting variables on hosts. So those two parts are always required.
Don't have to check for the existence of those in the function.  If they
aren't set to iterables then it should be up to the calling code to
handle it.
2017-07-14 07:03:26 -07:00
Brian Coca
84367eacef added ansible_remote_tmp inventory var (#26754)
* added ansible_remote_tmp inventory var

* fixed horrible crime of no space after #
2017-07-13 15:31:10 -07:00
Pilou
2a92120ffa INI inventory plugin: add documentation about variable types (#25798)
* INI inventory: check variable types
* INI inventory: add doc about variable types

Fixes #25784
2017-07-13 12:04:20 -07:00
Ricardo Carrillo Cruz
5719687920 Split net_lldp into net_lldp (service) and net_lldp_interface (#26757) 2017-07-13 16:38:15 +02:00
Ricardo Carrillo Cruz
9e819eeee8 Add net_lldp platform agnostic module (#26738) 2017-07-13 12:02:45 +02:00
Peter Sprygada
41c1457823 adds put_file and fetch_file implementations for cliconf plugin (#26728) 2017-07-13 09:21:02 +05:30
Brian Coca
a5007f2f88 Namespaced facts fixes (#26615)
* shorten warning on reservd fact collision

also remove ansible_ from namespaced facts for vars manager
handle str conversion errors
use tuple to avoid iterator errors
version added added

* only modify final one

* removed ansible_ removal
2017-07-12 19:27:05 -04:00
Brian Coca
c2326aef01 dont show failed key on debug 2017-07-12 14:26:32 -04:00
Ken Celenza
31b6ac896d Kc update ip filter (#26566)
* add first, last and next usable

* add usable ip filters

* add size usable, range usable and wildcard

* add ip prefix and netmask filter

* add network formatting and check if ip in subnet

* clean up order, add comments

* fix pep8

* update format by index

* clean up and updates from jmcgill298
2017-07-12 17:17:58 +01:00
Michael Scherer
1e5cfd16fc Fix typo in documentation for inventory ini plugin (#26643) 2017-07-12 14:00:40 +01:00
Tomas Tomecek
5b898a7732 add connection plugin for buildah (#26170)
* add connection plugin for buildah

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

* fixup

 * create a method to invoke buildah
 * mount container filesystem persistently so we can access it
   during put and fetch
 * use copyfile function for copying files

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

* revert tests

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

* fixup

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

* progress

(this will be squashed into a single commit before merge)

 * add docs for the conn plugin
 * fix issue invoking the integration tests
 * add a way to invoke commands inside the container as a different user

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

* fix shellcheck warning

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
2017-07-11 22:36:25 -04:00
Brian Coca
035b4b95b0 booleanize value for ternary 2017-07-11 13:05:27 -04:00
Manuvaldi
3ff67fc217 hashi_vault module - Add verify param to support ssl Vault (#25159)
* Fix conflic with HVAC library check

* Fix pep8 error

* hashi_vault add validate_certs parameter
2017-07-11 12:17:04 -04:00
Karim BEN YOUSSEF
804f40f7a5 script: Add support for chdir argument 2017-07-07 09:40:02 -07:00
Adrian Likins
1cad0074f5 code cleanup and error improvement for hashi_vault (#17824)
Use standard import error handling.
Make error messages more specific.
Use more python idiomatic code.
2017-07-07 10:17:18 -04:00
Toshio Kuratomi
5ceabe939d Fix the pause module for python3
* On python3, stdin goes through a buffer that translates from raw bytes
  to text.  this interferes with pause as it (1) performs universal
  newline conversion and therefore '\r' is turned into '\n' and (2) the
  buffering prevents us from getting the typed characters immediately
  (possibly a python3 bug?)  Using the raw byte stream that's behind the
  text decoder fixes these problems.

Unrelated cleanups:
* Use to_text instead of str for conversion into strings to avoid possible tracebacks
* Use either \r or \n as the end of a line.

Fixes #26278
Resolves #26446
2017-07-06 19:21:42 -07:00
Ricardo Carrillo Cruz
83253cd8f0 Add net_l3_interface vendor neutral module (#26484)
* Add net_l3_interface vendor neutral module

* Fix pep8 issue
2017-07-06 21:02:25 +02:00
Ricardo Carrillo Cruz
5acebc124a Add vendor neutral parameter fail_on_missing_module (#26482)
By default, the vendor neutral modules will just go on if no
implementation module is found.
If user specifies the task argument fail_on_missing_module and
sets it to True, then we bail out the play early and report that
to the user.
2017-07-06 14:10:29 +02:00
Ricardo Carrillo Cruz
8b0e052b88 Add missing net_linkagg action plugin (#26477) 2017-07-06 11:12:05 +02:00
James Mighion
e785cc53db Adding stderr regex catch for iosxr commit failures. Adding error handling after executing commit command. (#26462) 2017-07-06 09:19:24 +05:30
Brian Coca
2a041d10d2 better backwards compat handling of status
restored 'rc' inspection but only when failed is not specified
removed redundant changed from basic.py as task_executor already adds
removed redundant filters, they are tests
added aliases to tests removed from filters
fixed test to new rc handling
2017-07-05 21:44:00 -04:00
Toshio Kuratomi
cf5fb0acdf Do not escape backslashes when using the template lookup plugin
This brings the lookup plugin inline with what the template module does.

Fixes #26397
2017-07-05 16:39:20 -07:00
Jean-Frédéric
23041c3b6c Fix error with meta: clear_facts (#26406)
Using `meta: clear_facts` was failing with
`coercing to Unicode: need string or buffer, Host found`

This applies the same fix as 3101e24.

Fixes #26405
2017-07-05 19:20:18 -04:00
Brian Coca
02311604c9 removed logger from paramiko
(cherry picked from commit 9b7dfd5945ec1ab329e288493b6962d4e8c33fa0)
2017-07-05 18:09:40 -04:00
Nathaniel Case
28c6b226c7 ansible-connection Python3 fix (#26441)
* Fix a number of unicode <-> bytes mismatches

* Return socket_path as text, not bytes

* Docstring run()
2017-07-05 14:07:26 -04:00
Ganesh Nalawade
e612d0be12 Add net_l2_interface platform agnostic module (#26410)
* Add net_l2_interface platform agnostic module

*  net_l2_interface module definition
*  net_l2_interface action plugin

* Fix CI issue
2017-07-05 11:15:19 +05:30
Matt Davis
0ee46cb0df converted become runas to explicit CreateProcessWithLogonW (#26378)
* fixes become_method: runas for unprivileged users
* sets permissions on tempdir appropriately
* allows automatic system environment generation for new token (old Process.Start way prevents this)
* add basic become runas tests
2017-07-03 17:46:27 -07:00
Brian Coca
8f758204cf correct, cleanup & simplify dwim stack (#25956)
* correct, cleanup & simplify dwim stack

latlh chIS logh HeS qar wej chel laD
better errors
update find_file to new exception

* addressed latest comments

* test should not use realpath as it follows symlink

this fails when on OS X as /var is now a symlink to /private/var
but first_found was not supposed to follow symlinks
2017-07-03 15:27:53 -04:00
Ganesh Nalawade
11e72d495d Add net_logging platform agnostic module and junos implementation (#26316)
* Add net_logging platform agnostic module and junos implemenatation

*  net_logging platform agnostic module
*  junos implemenatation of logging module
*  net_logging integration test
*  junos_logging integration test
2017-07-03 15:31:59 +05:30
Brian Coca
4361659bee correctly deal with encoding in errors
fixes #26256
2017-06-30 14:40:38 -04:00
Toshio Kuratomi
6908038036 Fix synchronize setting user variable
The user variable stores whether we need to set user@ in our connection
string.  It's now being used at the toplevel of the run() method so the
default needs to be calculated further up the stack

Fixes #24910
2017-06-28 17:41:01 -07:00
Tomas Tomecek
2683c36a01 plugins,action: log stderr output if verbose (#25734)
otherwise ansible eats out important error messages

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
2017-06-28 15:55:39 -04:00
David Moreau-Simard
9fdd07fba8 Provide the list of files that were included by include_vars
include_vars will now also return a key 'ansible_included_var_files'
which contains the list of files that were successfully loaded.
This is useful information and, amongst other things, a way for users
to know exactly what files were included when debugging their
playbooks.
This also allows us to improve the integration tests around
include_vars.
2017-06-28 15:25:45 -04:00
Ganesh Nalawade
3bd8dbb53d Add support for netconf network discovery (#25435)
* Add support for netconf network discovery

* Fix unit test failure
2017-06-28 11:07:38 +05:30
Matt Davis
36ad934156 re-enable non-pipelined mode for Powershell (#25012)
* fixes #23986
* fixes 3rd-party Windows connection plugins that don't support pipelining (eg awsrun)
2017-06-26 22:58:09 -07:00
Ricardo Carrillo Cruz
ffaf6c4a86 Add new net_static_route platform agnostic module (#25455)
* Add new net_static_route platform agnostic module

* Add mask parameter and according examples

* Add required true to required params

* Add static routes collections example
2017-06-26 11:23:33 +02:00
Kassian Sun
de4a8b83df Fix up integer type detection on python3 (#25985) 2017-06-22 08:48:22 -04:00
Abhijeet Kasurde
e8072ee616 Misc typo for Ansible
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-06-21 11:19:41 -04:00
Brian Coca
d79315e19e avoid winrm errors when not connecting to winrm (#25921) 2017-06-21 01:13:06 -07:00
Brian Coca
0bbf0e8d19 adjust script caching to processed data
also added filename to loading for better errors
2017-06-20 14:27:53 -04:00
Brian Coca
e2cf3b6e7a fixed cg signature 2017-06-19 19:08:36 -04:00
Brian Coca
9d8721dbde properly track delegated host in loops
ansible_host can be pulled from inventory and not match inventory_hostname,
this can "loose" vars to a new host named by ansible_host vs the delegated host
fixes #25770
2017-06-19 17:50:52 -04:00
Brian Coca
e08f068dca ensure proper typing of path, cause py3 listdir
since now output is based on input, we need to make sure its always same
fixes #25856
2017-06-19 16:35:50 -04:00
Peter Sprygada
97e24dc317 fixes issue with SIGALARM call in network_cli (#25832)
The alarm_handler method was calling a method close_shell() that doesn't
exist.  This updates the alarm handler to call the current method
close()
2017-06-17 15:48:30 -04:00
René Moser
6feee18292 host_group_vars: skip files having blacklisted ending (#25730)
* vars: skip backup files

* vars: extend doc for ignored files
2017-06-16 18:45:58 +02:00
Ganesh Nalawade
2ff464c949 Add net_interface declarative module (#25766)
* Add net_interface declartive module

*  Add net_interface module
*  Add junos_interface implementation module
*  Other minor changes

* Add integration test

*  Integration test for net_interface
*  Integration test for junos_interface

* Fix CI failures

* Documentation changes
2017-06-16 22:12:50 +05:30
Rene Moser
46c6eb37d0 vars: fix typo s/path/spath 2017-06-16 17:17:38 +02:00
Rene Moser
43468b825d vars: fixed missing path in splitext() 2017-06-16 10:08:14 -04:00
Matt Clay
3eea649cbb PEP 8 fixes. 2017-06-15 15:54:05 -07:00
Brian Coca
13a1111dde tighter host/group_var discovery
now only loads nonext or valid yaml extensions
only first file gets loaded
directories must match name (initially) so no name.yml is loaded anymore
2017-06-15 12:35:18 -04:00
Brian Coca
a54666d31e now host list removes trailing/leading spaces
othewise hostnames were just 'fun'
2017-06-14 16:57:00 -04:00
Brian Coca
077ca3e07d skip hidden files for {host,group}_vars
fixes #25712
2017-06-14 15:48:42 -04:00
Brian Coca
74842adc07 1st part of ansible config, adds ansible-config to view/manage configs (#12797)
* Start of ansible config project

moved configuration definitions to external yaml file vs hardcoded
 * updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
 * prints green for default/unchanged and yellow for those that have been overriden
 * added list action to show all configurable settings and their associated ini and env var names
 * allows specifying config file to see what result would look like
 * TBD update, edit and view options

removed test for functions that have been removed

env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling

updated as per feedback

* pep8
2017-06-14 11:08:34 -04:00
Ganesh Nalawade
b69c7f50d0 Add junos_vlan platform agnostic module (#25652)
* Add junos_vlan platform agnostic module

*  junos_vlan implementation

*  integration test for junos_vlan

* Minor fixes

* Fix Integration test failure
2017-06-14 14:23:52 +05:30
Brian Coca
4ee348e564 draft constructed groups plugin (#23374)
* draft constructed groups plugin

can use fact cache as vars source

* pw4er5e
2017-06-13 10:46:40 -04:00
Brian Coca
c4169d93e8 add validation to fetch for source/dest
also added missing display
2017-06-13 10:37:14 -04:00
Toshio Kuratomi
90b1d780eb Fix redis cache for python3
According to the redis-py docs, zrank will return the 0 based index for
the value in the sorted set.  So the logic here wasn't right to begin
with (It just means that a value at the 0-th position would never show
up as cached).  Need to compare against None to know if the value
exists in the cache.

https://redis-py.readthedocs.io/en/latest/#redis.StrictRedis.zrank

Fixes #25590
2017-06-13 09:50:46 -04:00
Brian Coca
498aea8acc Revert "Add log_only to debug messages (#25545)"
This reverts commit 9e8cc26720.
2017-06-13 09:29:18 -04:00
Ganesh Nalawade
9e8cc26720 Add log_only to debug messages (#25545)
Fixes #25544
When `debug` is enabled the debug messages triggered
from `bin/ansible-connection` should be logged only to file
and not on stdout.
2017-06-13 18:41:33 +05:30
Abhijeet Kasurde
b89cb95609 Fix spelling mistakes (comments only) (#25564)
Original Author : klemens <ka7@github.com>

Taking over previous PR as per
https://github.com/ansible/ansible/pull/23644#issuecomment-307334525

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-06-12 07:55:19 +01:00
Matt Martz
23eebb62e6 Convert docker cmd output to native, early 2017-06-10 07:09:04 -07:00
Matt Clay
a18cc45778 Fix PEP 8 issue. 2017-06-10 14:07:15 +08:00
Brian Coca
5717f4f161 updated cache descriptions 2017-06-09 23:48:47 -04:00
Brian Coca
cdcbd84088 refactor of virtualbox plugin, fixed cache use 2017-06-09 23:48:47 -04:00
Ricardo Carrillo Cruz
72e65e4290 Platform agnostic module for VRFs (#25383)
* WIP VRF platform agnostic module

* Fixed examples refering vlans instead of vrfs

* Add integration tests
2017-06-09 13:33:11 +02:00
Brian Coca
8badfcd7fe make sure ini handles children no matter the order
fixed 'pending declarations' assumption that children only have 1 parent
fixes #25488
2017-06-08 10:37:19 -04:00
Matt Martz
13c070948b Fix pep8 issues in profile_roles (#25458)
* Fix pep8 issues in profile_roles

* Fix reduce import
2017-06-07 21:20:59 -05:00
Brian Coca
2e920d724d fixed import, had moved 2017-06-07 19:11:45 -04:00
Tennis Smith
1d15ca739f feat: add role profiler (#20092)
* feat: add role profiler

* fix: use 2.0 api version
2017-06-07 18:38:32 -04:00
Brian Coca
88f0e0b882 removed spurious warning (traling ,)
also removed unused improt
2017-06-07 15:16:47 -04:00
Matt Martz
004e99316c Allow template files to be vaulted (#22951)
* Allow template files to be vaulted

* Make sure to import exceptions we need

* get_real_file can't take bytes, since it looks specifically for string_types

* Now that we aren't using open() we don't need b_source

* Expand playbooks_vault docs to include modules that support vaulted src files

* Add vaulted template test
2017-06-07 14:16:03 -04:00
James Cammarata
483df9c5f8 Imports and includes (#25399)
Initial commit to split includes into static imports/dynamic includes

This implements the new include/import syntax for Ansible 2.4:
* include_{tasks,role,variables} = dynamic
* import_{playbook,tasks,role} = static

The old bare `include` will be considered deprecated, as will any use of the `static: {yes|no}` option.

This also adds docs for import/include and reorganizing the "Playbook Reuse" section of the documentation.
2017-06-06 16:39:48 -05:00
Trishna Guha
f6a4803669 New module: platform agnostic way to manage local users on network devices (network/net_user.) (#25259)
* Add net_user platform agnostic module

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* Integration test for net_user

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* modify eos_user module to support name param as alias to username

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* Test collection of users

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
2017-06-06 20:05:08 +05:30
Ricardo Carrillo Cruz
1476e30c26 Add net_vlan platform agnostic module (#25252)
* Add net_vlan platform agnostic module

* Add vlan_id param

* Fix documentation and examples

* Remove whitespace
2017-06-06 11:45:02 +02:00
Ganesh Nalawade
6215922889 Add support for cliconf and netconf plugin (#25093)
* ansible-connection refactor and action plugin changes
* Add cliconf plugin for eos, ios, iosxr, junos, nxos, vyos
* Add netconf plugin for junos
* Add jsonrpc support
* Modify network_cli and netconf connection plugin
* Fix py3 unit test failure
* Fix review comment
* Minor fixes
* Fix ansible-connection review comments
* Fix CI issue
* platform_agnostic related changes
2017-06-06 13:56:25 +05:30
Brian Coca
cf59f9765e fix script inventory plugin
fixes #25371

there is still better ways to do this, but this will 'unbreak' devel for now
2017-06-05 18:26:11 -04:00
Matt Martz
99a30f8fdb Python3 fix for hashing in inventory plugins 2017-06-02 13:45:50 -04:00
Trishna Guha
e90f1d6449 net_command platform agnostic module (#25249)
* net_command platform agnostic implementation

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* Add net_command platform agnostic module

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* Add integration test for net_command module

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* fix yaml issue
2017-06-02 22:03:27 +05:30
Ricardo Carrillo Cruz
955dc6e250 Add net_banner platform agnostic module (#25245)
* Add net_banner platform agnostic module

* Add integration tests for net_banner

* Remove default from required param

* Remove skip/python3 from net_banner aliases
2017-06-02 17:01:08 +02:00
Ricardo Carrillo Cruz
64add28657 Platform agnostic net_system module (#24953)
* Platform agnostic net_system module

Also refactor the action network plugins for better code re-use

Still more refactoring to do once the connection plugin work is complete

* Replace importlib for imp

importlib is not available on 2.6, so we need to stick to imp

* Load action plugin via module metadata

* Better error message if no implementation is found

Now the plugin will show the module name and the network OS in the
error message

* Fix typo on documentation author line

* Fix pep8 issues

* Add missing options key on doc string and stringify version

* Return None in case module has no metadata

* Read module metadata only if it's a python module

Check for module suffix, if it's .py then read metadata.
Otherwise this fails on non-python modules, like Windows PS for example.

* Read metadata variable only if it's a python module

Fix referencing a variable before assignment

* Add action_handler to validate_modules metadata schema

* Pull metadata with plugin_docs get_docstring

Using load_source from PluginLoader is troublesome, it is not guaranteed
a module may be importable at the controller, e.g. if a module depends
on module_utils functions it won't work, because module_utils is not
in the sys path.
Rather than putting that module dependencies introspection, just
use plain parsing like plugin_docs get_docstring does as we only care
about reading ANSIBLE_METADATA.

* Add platform agnostic group of groups for integration tests

This will be the target for platform agnostic integration tests.

* Add integration tests for net_system

* Switch to action plugin inheritance from metadata driven action handler

As the metadata action driven action handler work is being worked on
on its standalone proposal+PR, let's just go back to have one
action handler per platform agnostic module.
Those action plugins will inherit from net_base.

* Add blank line to fix pep8

* Add aliases file to net_system integration test

This will avoid CI failure

* Fix integration tests for net_system

* Give more precedence to task network_os over inventory network_os
2017-06-02 14:06:38 +02:00
Dag Wieers
5553b20828 Collated PEP8 fixes (#25293)
- Make PEP8 compliant
2017-06-02 12:14:11 +01:00
Abhijeet Kasurde
07be5f2b67 spelling fixes (non-trivial, changing messages) (#25094)
Multiple spell fixes in single commit.
Taking over existing PR as per comment -
https://github.com/ansible/ansible/pull/23645#issuecomment-304312275

Original Author: klemens <ka7@github.com>

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-06-01 10:45:19 +01:00
piotrsmolinski
c3ffc49914 reset_connection fixes: #23621 (#25211)
* moved the logging statement

moved the logging statement before the actual action

* added status code check

In the existing implementation when the ssh command fails the command
result is silently discarded. It hides the fact that the disconnection
did not go as expected. Effectively the intended action was not
successful, but the play continues.

* Revert "added status code check"

This reverts commit fe2eb2ae4aeb4812fa2f59ccdfabc9efc677e657.

* added command status code check

In the existing implementation the command is checked for the success.
As a result failed execution is silently discarded. The change tests for
return code and fails if it did not work.
2017-05-31 13:40:57 -04:00
Dag Wieers
c13b1a718a plugins/terminal/: PEP8 compliancy (#25207)
- Make PEP8 compliant
2017-05-31 18:28:03 +01:00
Brian Coca
caf5868ec9 removed debugging prints 2017-05-31 13:08:50 -04:00
Brian Coca
798d05ea41 corrected object with columns 2017-05-31 13:03:10 -04:00
Brian Coca
5667603a27 fixed format 2017-05-31 12:52:44 -04:00
Brian Coca
62a434141e ensure display is terminal sensistive
fixes #18248 by using display width intead of hardcoded length
2017-05-31 11:15:45 -04:00
Dag Wieers
0e160d5c7e Ensure exit_json returns failed = False
This is required for modules that may return a non-zero `rc` value for a
successful run, similar to #24865 for Windows fixing **win_chocolatey**.

We also disable the dependency on `rc` value only, even if `failed` was
set.

Adapted unit and integration tests to the new scheme.
Updated raw, shell, script, expect to take `rc` into account.
2017-05-30 14:56:31 -07:00
Brian Coca
5d572055b3 fixed script plugin
fixes #25130
2017-05-30 11:10:43 -04:00
Brian Coca
bc403934ef properlly report plugin name 2017-05-30 11:10:43 -04:00
Brian Coca
fd6302bcb8 only set ansible_host if i have data 2017-05-28 11:19:44 -04:00
Brian Coca
68936d0530 updated vbox docs 2017-05-28 11:14:03 -04:00
Brian Coca
21da284f5d added ability to create vars from vbox properties 2017-05-28 10:55:50 -04:00
Brian Coca
5cd0074831 fixed virtualbox plugins, expanded docs 2017-05-28 10:37:18 -04:00
Brian Coca
46a3e6088b fixed shared 'compose' method for inventory plugins 2017-05-28 10:37:06 -04:00
Brian Coca
110fd917d6 adding caching to script plugin 2017-05-25 15:37:39 -04:00
QijunPan
e86279cee4 Contributing new /lib/ansible/plugins/action/ce.py module to manage HUAWEI data center CloudEngine switch (#21645)
* commit plugins action ce module

commit plugins action ce module

* update plugins/action/ce.py

update plugins/action/ce.py

* update action ce.py

update action ce.py

* fix CI issues

* update ce.py

* add ce action
2017-05-25 17:33:57 +01:00
Brian Coca
7c325470e2 Pepe Ate 2017-05-25 11:38:53 -04:00
Brian Coca
f23920aa5e optimize file finding 2017-05-25 11:38:53 -04:00
Benjamin Schwarze
5668c97850 enable su support for docker connection 2017-05-24 16:53:08 -07:00
Brian Coca
40b222867f ensure fullpaths on vars files (#24990)
fixes #24970
now correctly picks up group/host vars inside group/host named directories
2017-05-24 12:50:05 -04:00
John R Barker
c158705c48 Fix pep8 issue
lib/ansible/plugins/vars/host_group_vars.py:74:41: E261 at least two spaces before inline comment
Caused by daef6f0911
2017-05-24 16:53:02 +01:00
Ganesh Nalawade
825d9df5ea Add eos changes for Python3 (#24600)
* eos python3 changes

* changes to convert response from byte to text

* Add dellos6 python3 changes
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431

* Fix py3 prompt issue for invalid show command
* Fix review comments
* Add generic fix for error prompt in py3
* Fix CI issue
* Fix network_cli unit test failure
2017-05-24 19:40:38 +05:30
Brian Coca
daef6f0911 allow empty files in host/group_vars
also remove unused import
fixes #24963
2017-05-24 09:32:34 -04:00
Brian Coca
6dc2a6ce8b fixed docs for host list examples 2017-05-23 22:53:12 -04:00
Joseph Callen
8ca836ad14 Fix script inventory plugin 2017-05-23 22:25:10 -04:00
Nathaniel Case
9737c6b90d Python3 chokes on casting int to bytes (#24952)
But if we tell the formatter that the var is a number, it works
2017-05-23 15:58:37 -07:00
Brian Coca
8f97aef1a3 Transition inventory into plugins (#23001)
* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local
2017-05-23 17:16:49 -04:00
Brian Coca
77be724913 fixed extension matching
and name is valid for all
2017-05-23 15:50:35 -04:00
Brian Coca
6fa5dbdb5c fixed pep8 issues 2017-05-23 15:39:22 -04:00
Brian Coca
e0e94a4d02 reinstated to legacy
(cherry picked from commit a6484da0f11d03056b446526ef1cf78f968ae3ee)
(cherry picked from commit f23e766efa859b25557537d5098de557e85120cf)
2017-05-23 10:40:44 -04:00
Abhijeet Kasurde
f9b836a901 Include error exception in AnsibleError
- Use to_native instead of str

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-05-23 10:19:14 -04:00
Matt Martz
4ac135c1b5 Fix logic surrounding copy and remote_src, remote_src is preferred, make copy action plugin only. Fixes #23591 (#24732) 2017-05-22 17:19:56 -07:00
Ganesh Nalawade
7563d93901 Add nxos changes for Python3 (#24602)
* Add nxos changes for Python3

Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431

* Fix CI issues

* Fix review comment

Replace surrogate_or_strict with
surrogate_then_replace as per review
comment os PR #24601
2017-05-20 01:45:53 +05:30
Michael
b65ebf3519 Fixed import of urlencode and pathname2url from urllib for python3 (#24424) 2017-05-19 12:22:16 -05:00
Trishna Guha
4782a4e62f Make host required field and minor refactor (#24534)
* Make host required field and minor refactor

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* ansibot pep8 legacy file

* example doc update
2017-05-19 10:37:01 -06:00
QijunPan
6352a54f19 Contributing new lib/ansible/plugins/terminal/ce.py module to manage HUAWEI data center CloudEngine switch (#21653)
* commit plugins terminal ce.py module

commit plugins terminal ce.py module

* update plugins/terminal/ce.py

update plugins/terminal/ce.py

* deal with pep8 error
2017-05-19 08:43:37 +01:00
QijunPan
ad85f2b483 Contributing new /lib/ansible/plugins/action/ce_template.py module to manage HUAWEI data center CloudEngine switch (#21649)
* commit plugins action ce_template.py

commit plugins action ce_template.py

* deal with pep8 error
2017-05-19 08:42:55 +01:00
QijunPan
627ac2493d commit plugins action ce_config module. (#21648)
commit plugins action ce_config module.
2017-05-19 08:42:36 +01:00
Matt Martz
1eed6c5b3d Convert some more base64 strings to text for winrm, to support async and become (#24796) 2017-05-18 16:06:51 -07:00
James Cammarata
5728bd9065 Merge branch 'modules_issue_1568' of https://github.com/ajdecon/ansible into ajdecon-modules_issue_1568 2017-05-18 02:28:08 -05:00
Matt Martz
e7d8ebf080 Base changes required to allow winrm to work on py3 (#24744) 2017-05-17 14:26:32 -07:00
James Cammarata
f217dae938 Also check the iterator for failed host status in linear strategy
When only looking at the failed state of the TaskResult, certain failures
cause the linear strategy to fail out sooner than it should and not execute
the always portion of blocks.

Fixes #24301
2017-05-17 13:41:29 -05:00
Ganesh Nalawade
38eeeb755d Add ios changes for Python3 (#24601)
* Add ios changes for Python3

Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
pep8 fixes

* Fix CI issues

* Fix review comment
2017-05-17 20:00:52 +05:30
Ganesh Nalawade
e52d06b401 Add dellos9 changes for Python3 (#24607)
* Add dellos9 changes for Python3

Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431

* Minor changes
2017-05-17 00:36:52 +05:30
Ganesh Nalawade
a883635aeb Add dellos6 changes for Python3 (#24606)
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
2017-05-17 00:36:32 +05:30
Ganesh Nalawade
442df927f1 Add dellos10 changes for python3 (#24608)
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
2017-05-17 00:36:15 +05:30
Ganesh Nalawade
a3c689bf0d Add sros changes for Python3 (#24604)
* Add sros changes for Python3

Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431

Code cleanup
pep8 fixes

* Fix CI issue
2017-05-17 00:33:36 +05:30
Ganesh Nalawade
16d610c1de Add vyos changes for Python3 (#24603)
* Add vyos changes for Python3

Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
pep8 fixes

* Fix CI issues
2017-05-17 00:32:59 +05:30
Ganesh Nalawade
f4e1dc503b Add iosxr changes for Python3 (#24605)
* Add iosxr changes for Python3

Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431

* Fix CI issue
2017-05-17 00:32:41 +05:30
James Cammarata
e5cd675b38 Use delegated vars for the delegated host name
In _process_pending_results (strategy/__init__.py), we were using the delegate_to
field directly from the original task, which was not being templated correctly.
As an alternate to #23599, this patch instead pulls the host name out of the delegated
vars passed back in the task result dictionary to avoid having to re-template things.

Fixes #23599
Fixes #20508
2017-05-16 10:24:42 -05:00
Toshio Kuratomi
84a59e472b Be explicit about pickle protocol and encoding (#24454)
On Python3 and Python2 use pickle slightly differently so we need to be
explicit about some things.

If pickles could be shared between python2 and python3, as in
ansible-connection and the pickle cache, we need to specify the protocol
to use when dumping and the encoding to use for byte strings when
loading.

The dumping protocol needs to be no higher than 2 as python-2 only
supports up to protocol 2.  The encoding should usually be 'bytes' so
that python2 str type becomes python3 bytes type.  However, doing this
means that we must make sure that the objects being serialized properly
make their strings into text strings except when they're supposed to be
bytes.  If strings are improperly byte strings, they may cause
tracebacks on the receiving end
2017-05-12 18:04:48 -07:00
Toshio Kuratomi
d834412ead Fix for persistent connection plugin on Python3 (#24431)
Fix for persistent connection plugin on Python3.  Note that fixes are also needed to each terminal plugin.  This PR only fixes the ios terminal (as proof that this approach is workable.)  Future PRs can address the other terminal types.

* On Python3, pickle needs to work with byte strings, not text strings.
* Set the pickle protocol version to 0 because we're using a pty to feed data to the connection plugin.  A pty can't have control characters.  So we have to send ascii only.  That means
only using protocol=0 for pickling the data.
* ansible-connection isn't being used with py3 in the bug but it needs
several changes to work with python3.
* In python3, closing the pty too early causes no data to be sent.  So
leave stdin open until after we finish with the ansible-connection
process.
* Fix typo using traceback.format_exc()
* Cleanup unnecessary StringIO, BytesIO, and to_bytes calls
* Modify the network_cli and terminal plugins for py3 compat.  Lots of mixing of text and byte strings that needs to be straightened out to be compatible with python3
* Documentation for the bytes<=>text strategy for terminal plugins
* Update unittests for more bytes-oriented internals

Fixes #24355
2017-05-12 09:13:51 -07:00
Jan-Piet Mens
0e0e5097a9 Optionally return raw data from URL lookup using split_lines=False 2017-05-12 11:02:49 -04:00
Toshio Kuratomi
548cacdf6a Fix template not showing a diff with a directory
Template can take a directory as the destination.  When that's the case,
we need to diff between the source and the file inside of the directory.
That happened when the directory was specified with a trailing slash but
not when it was specified on its own.  This change fixes that.

Fixes #24413
2017-05-11 20:37:31 -07:00
James Cammarata
78478e80ea Avoid default serialization of classes in dump_attrs
For playbook base objects, when dumping attributes via dump_attrs() an
attribute like loop_control is a class. Using the default serialization
for these is slow and consumes a lot of memory. Since LoopControl is also
based on the Base class, we can use serialize() instead and save a lot of
resources.

This also adds a from_attrs() complimentary method to nicely turn the
dumped attrs back into proper field attributes.

Fixes #23579
2017-05-11 16:28:07 -05:00
Brian Coca
c50cf22d52 deal with null/none connections
fixes #23621
pushed 'connection resolution' to play_context
override fieldattribute getter
2017-05-11 15:46:14 -04:00
wiso
bb90f5d220 netconf_config: support new src attribute (new Action Plugin) (#24337)
* Create netconf_config.py

To support new attribute 'src' for the netconf_config module.
Cross-check with #24323

* Update netconf_config.py

* Update netconf_config.py
2017-05-11 18:43:44 +05:30
Matt Davis
98ff93b2db fix spurious warnings with ansible_winrm_kinit_X args (#24380)
* added to pywinrm arg whitelist
* clarified error text on kinit_mode error
* fixes #23822
2017-05-08 11:27:25 -07:00
Brian Coca
b731c2d809 include_vars to delegated only if delegate_facts (#24259)
* include_vars to delegated only if delegate_facts

fixes #24172

* since code is now same, simplified
2017-05-03 16:30:20 -04:00
Toshio Kuratomi
a0dfa8616a Make fetch default to fail on errors
Fixes #23501
2017-05-03 09:37:31 -07:00
Matt Martz
040fb4435a Fix password prompt matching (#24081)
* Fix password prompt matching

* Add some tests for check_password_prompt

* Prevent pep8 line ends with a space error
2017-05-01 11:18:15 -04:00
Nick Piper
403c142750 Minor typo correction varibles -> variables
No impact as variable wasn't used.
2017-05-01 08:44:48 -04:00
Toshio Kuratomi
1c05ed7951 Fix circular import with unsafe_proxy, template, and vars
template/__init__.py imported unsafe_proxy from vars which caused
vars/__init__.py to load.  vars/__init__.py needed template/__init__.py
which caused issues.  Loading unsafe_proxy from another location fixes
that.
2017-04-28 15:35:30 -07:00
Nathaniel Case
f9ec06d1dc Print specific error for BadHostKeyException (#24120) 2017-04-28 13:57:32 -04:00
Erwin Lang
b729b35e32 synchronize: Fix (delegated) local rsync
Makes delegated local rsync work even if ansible_host or ansible_ssh_host is set.
Makes local rsync work when no ssh is installed.
2017-04-27 09:33:26 -07:00
Nathaniel Case
689b93bf14 Junos_config unicode (#23369)
* Try to handle unicode output more sensibly

* Appears I'm getting latin1 instead

Ugh.
2017-04-27 11:44:26 -04:00
Toshio Kuratomi
bffccb5396 fetch idempotence test and deprecate validate_md5
Added an integration test for fetch module idempotence.  (Testing
that validate_checksum is doing what it's supposed to is harder as we'd
have to create a race condition with the downloaded data to trigger it.
Probably need to make that a unittest eventually).

Also give a deprecation message to the validate_md5 parameter so that we
can eventually get rid of it.
2017-04-26 12:00:01 -07:00
KeepZero
98e7d4b49d Set validate_checksum default yes of fetch to match the doc 2017-04-26 09:27:06 -07:00
Brian Coca
e29dc49a49 moved to exceptions for basic skip/fails
better handling of checkmode and async
fix test to follow new flow control
2017-04-25 14:48:56 -04:00
Trishna Guha
f354bd1eab Set terminal width for network *_command modules to avoid paging (#23728)
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
2017-04-25 14:10:39 +05:30
Brian Coca
35e03cb140 p8p 2017-04-21 23:36:44 -04:00
Brian Coca
27185f44b0 fix environment populated detection
fixes #23764
2017-04-21 18:21:01 -04:00
Brian Coca
3358abcf49 Add a new filter: strftime. Use the well known function to format a date output. (#23832)
(cherry picked from commit 3f5b304fc28b6c34df9ea6a4c3531dc422ce198b)
rebased for @yannig
2017-04-21 11:48:45 -07:00
Ricardo Carrillo Cruz
8517fbf936 Remove catch-all regex on IOS terminal plugin (#23858)
We have a list of specific messages that we scree-scrape and flag
them as legit errors.
However, we also have a catch-all regex that matches everything
starting with %.
That can cause issues on commands that return lines with that
character, like for example the 'crypto key generate'.

Fixes #23770
2017-04-21 15:30:23 +02:00
Nathaniel Case
0c2a0c7bce Eos port default values (#23829)
* Hook port back up for eos cli transport

* The default values have moved

* Why not have conditional defaults?
2017-04-20 15:58:45 -04:00
Brian Coca
fe9af903f3 script fails on async 2017-04-20 11:49:10 -04:00
Brian Coca
3965689328 finer grained failed/skipped on async for actions
fixes #23729
2017-04-20 10:58:50 -04:00
Brian Coca
3f62dfda92 nicer docs 2017-04-18 13:13:14 -04:00
Matt Martz
f82d95ae28 When become_method is su, self._play_context.prompt is a function. Fixes #23689 2017-04-18 13:01:18 -04:00
Brian Coca
586fcae398 tolerate 'batch' systems that mess with stdin (#23596)
* tolerate 'batch' systems taht mess with stdin

fixes #23541

* have pause on windows tolerate devnull

* tuplie
2017-04-18 11:39:49 -04:00
Brian Coca
1ef2b6b302 added mongodb lookup docs 2017-04-18 11:12:09 -04:00
Antoine Cotten
7c308b3f8c Fix nonexistent module_name in callbacks (#23587) 2017-04-17 12:16:19 -07:00
Ganesh Nalawade
89752cc34f Make logging consistent for network platforms (#23605) 2017-04-16 23:25:40 +05:30
Peter Sprygada
c2370f14dd fixes issue where network modules would ignore provider host value (#23589)
This addresses a problem where the action plugin would ignore the
remote_addr value for the host.  In this case, only the inventory values
for the hostname would be considered and populate the remote host
remote_addr value for the connection plugin.
2017-04-14 15:38:52 +05:30
Brian Coca
5b7e8a7bd6 centralize ansible_managed and other template vars
now template lookup supports these again.
2017-04-13 11:03:11 -04:00
Brian Coca
f7566ef1f1 removed extra 'changed' from debug output 2017-04-13 11:03:11 -04:00
Senthil Kumar Ganesan
c86feeaed9 Error string:Regular expression updates (#23426) 2017-04-13 09:12:32 -04:00
Nathaniel Case
86c714548f Redo #23418 (#23425)
* Check for provider values inside check_args of respective network code

* Partial revert of b9ee5aa

The no_log change is okay, but take out the action result munging
2017-04-07 19:22:17 -04:00
Toshio Kuratomi
2fdb8e7f90 Fix a traceback with python3 and diff output
When retrieving file contents for diffing we need to get the contents as
binary.  Otherwise python3 will try to convert the file to text and fail
with non-decodable contents.

Fixes #23171
2017-04-07 14:35:50 -07:00
Chris Alfonso
d4bd54d3b8 Fixes #23388 Network provider masks too much (#23418)
* Fixes #23388 Network provider masks too much

* Fix trailing whitespace
2017-04-07 14:34:47 -06:00
Toshio Kuratomi
78ced5318f Fix for recursive copy slowness
Copy module was walking over files in subdirectories repeatedly (a
directory tree a few levels deep could bring the time spent into the
tens of minutes)

This was traced to the fix for this bug report: https://github.com/ansible/ansible/issues/13013

Fixed #13013 a different way and added an integration test to check for
regressions of #13013 as we optimize this code.

Fixes #21513
2017-04-07 12:41:42 -04:00
Strahinja Kustudić
9f0be5a556 Ignores ControlPersist broken pipe errors. Fixes #16731
Ansible will now automatically retry a connection if SSH returns an error:

mux_client_hello_exchange: write packet: Broken pipe

This is probably a bug in SSH, but because it's safe to retry this
connection there is no need for Ansible to fail because of it.
2017-04-07 09:06:28 -07:00
Matt Clay
48eeab8a53 Fix errors reported by pylint. (#23282)
* Fix pylint misplaced-bare-raise errors.
* Fix pylint return-in-init error.
* Fix pylint bad-format-character error.
* Fix pylint too-many-format-args errors.
* Fix pylint too-few-format-args errors.
* Fix pylint truncated-format-string error.
2017-04-06 16:58:16 -07:00
Ricardo Carrillo Cruz
3537b24742 Unhardcode the port and protocol on eos EAPI (#23350)
We were hard-coding the protocol, port and validate_certs on
eos EAPI via the action plugin.
Put defaults on the eos_argument_spec and pull those values from it.
2017-04-06 19:18:28 +02:00
John R Barker
13dc02d49d More network_debug_troubleshooting.html (#23335)
More "unable to open shell" -> network_debug_troubleshooting.html
2017-04-06 12:39:59 +01:00
John R Barker
529df8640b 'unable to open shell' -> direct to web help (#23267)
* 'unable to open shell' -> direct to web help

The "unable to open shell" error is returned for a number of different,
direct people to online docs (we we can update out of band of releases)
to guide them though the various solutions.

* fix pep8 errors
2017-04-06 10:11:28 +01:00
Matt Davis
8d291f91ee improve become_method: runas error handling (#23328)
Prescriptive errors for username/password issues and NTLM/Kerb auth failures, cleans up exception noise.
2017-04-06 01:34:30 -07:00
James Cammarata
a567c5de65 Only insert notified host if it's not already notified for certain cases
Fixes #18178
2017-04-04 16:03:29 -05:00
JP Mens
5a0b2fe3be dig lookup-plugin: add support for specifying rdclass (#10493)
* add support for specifying rdclass

* fix exception
2017-04-04 15:00:56 -04:00
Toshio Kuratomi
6bad4e57bd Migrate most uses of if type() to if isinstance()
Also convert those checks to use abcs instead of dict and list.

Make a sentinel class for strategies to report when they've reache the end
2017-04-04 08:17:20 -07:00
Senthil Kumar Ganesan
632270b913 Updated the Regex for promt matching (#23186) 2017-03-31 18:29:45 +01:00
Will Thames
ef0bc0aa52 Support check_mode for add_host
`add_host` doesn't really actually change anything - there's no
reason why it shouldn't work in `check_mode`.
2017-03-31 00:40:24 -04:00
Alex Willmer
f5aa9df1fd Include '/' & '.' when password_hash generates a new salt
The password_hash filter will generate a salt value if none is supplied.
The character set used by Ansible

(upper & lowercase letters, digits)

did not match that used by libc crypt

(upper & lowercase letters, digits, full stop, forward slash).

This resulted in a slightly smaller key space, and hence hashes would be
slightly easier to attack (e.g. by dictionary, brute force).
2017-03-30 12:33:39 -07:00
Senthil Kumar Ganesan
a0344acd78 Ansible 2.3 feature support for dellos6. (#23084)
* Ansible 2.3 feature support for dellos6.

- With the new Ansible 2.3 infra changes, the dellos modules doesn't work
  (the new infra changes are not backward compatible), so added the below
  changes support it.
- Added the new terminal plugin for DellOS6
- Added the new action plugin for DellOS6
- Modified the modules to work with the new infra.
- with that it adds support for DellOS6 Persistent Connection support.

* Remove pep8 confirming files from dellos6.py and dellos6_config legacy-files
2017-03-30 15:26:32 +02:00
Konstantin Suvorov
c1b3d6a51f Fix 'sequence' lookup shortcut syntax and documentation (#22989)
* fix 'sequence' lookup shortcut syntax and documentation

* Update playbooks_loops.rst

Minor edits for grammar and clarity.

* Update playbooks_loops.rst

Another tweak for clarity.
2017-03-29 19:11:19 -04:00
Peter Sprygada
88ce6fd273 restores pre 2.3 behavior with respects to keys (#23083)
This removes the requirement to configure look_for_keys=False and
restores the behavior to disable key lookup if no key was provided.
2017-03-29 15:12:18 -04:00
Toshio Kuratomi
6f77498700 Split on newlines when searching for become prompt
The fix for leading junk in sudo output: fee6e29 causes problems with
ssh + sudo.  On the initial connection using ControlPersist, the output
that we scan for the prompt contains both the command we're sending to
configure the prompt and the prompt itself.  The code in fee6e29 ends up
sending the password when it sees the line configuring the prompt which
is too early.

Switch to a version that splits on lines and then checks whether the
first or last line starts with the prompt to decide if it's time to send
the password.

Fixes #23054
References #20858
2017-03-29 12:11:10 -07:00
Victor Perron
bc44175d8d paramiko_ssh: fix crash upon pass prompt in py3
The pass prompt expects an answer and compares a `str` to a binary buffer, thus crashing.

It's an obvious fix to help transitioning towards Python3 and hopes it does not need a specific test.
2017-03-29 07:47:16 -07:00
Toshio Kuratomi
d3a1aea7c5 Fix a couple issues in synchronize with docker (#23047)
* Fix a couple issues in synchronize with docker

* Make the rsync_opts parse as a list using the same criteria as
  module_utils argumentspec parsing
* Do not quote arguments in the action plugin.  The module will quote as
  it knows whether it will invoke rsync with a shell or via exec.

Fixes #23046
2017-03-28 11:38:37 -07:00
Peter Sprygada
768cb437ab minor updates to network connection plugins (#23043)
* removes unused log() function in network_cli
* adds method comments to terminal plugin base
2017-03-28 12:57:18 -04:00
Senthil Kumar Ganesan
dd63dfcf1e Ansible 2.3 feature support for dellos9 and dellos10 (#22856)
* Ansible 2.3 feature support for dellos9 and dellos10
* Use Persistent Connection Manager

* Fix CI issue, revert the doc and metadata changes

* Reverted the meta_info (supported_by) to community from core

* Fixed the CI issues, use module_utisl.six and updated legacy-files
2017-03-27 19:32:57 +01:00
Peter Sprygada
ab4b8cb104 removes unused code in eos action plugin (#22986) 2017-03-27 00:50:55 -04:00
Peter Sprygada
3169cbd493 roll up of fixes for sros modules (#22972)
* fixes action handlers for sros
* fixes sros_config module execution to use AnsibleModule
* fixes sros_command module to use socket connection
* adds sros to constants
2017-03-25 10:35:15 -04:00
Peter Sprygada
6a414371a1 fixes issue where nxos module will fail due to KeyError (#22966)
Updates nxos action handler to handle deleting provider key if exists or
silently continuing if a  KeyError is raised.
2017-03-25 07:45:08 -04:00
Peter Sprygada
e93cdecef1 fixes log message in junos action handler (#22965) 2017-03-25 00:38:05 -04:00
Brian Coca
7b197d823e let user control "auto-de-vault" (#22739)
* added option to toggle off autodevaulting

* fixes per feedbazck
2017-03-24 15:39:25 -04:00
Matt Davis
e084e8809e force Windows to always use preamble-free UTF8 input encoding (#22934)
* fixes #15770
* When running under the UTF-8 codepage, Powershell subprocesses will fail (eg, Start-Job, others) if the input encoding is using the default BOM preamble. This fix forces it to use no preamble in leaf_exec and win_shell, and includes tests to verify that Start-Job works.
2017-03-24 00:02:39 -07:00
Dag Wieers
ac43a1bbbc Windows: Use the correct newline sequence for the platform (#21846)
This change to the template action plugin make template use the
platform's native newline_sequence for Jinja.

We also added the option `newline_sequence` to change the newline
sequence using by Jinja if you need to use another newline sequence than
the platform default.

This was previously discussed in
https://github.com/ansible/ansible/issues/16255#issuecomment-278289414

And also relates to issue #21128
2017-03-23 19:47:10 -07:00
Peter Sprygada
866f67e213 fixes a number of issues with nxos_facts (#22923)
* updates command runs to return warning if command fails
* fixes variable issues from recent refactoring
* removes provider from return of module
2017-03-23 22:35:15 -04:00
Matt Davis
73f50b4f9f fix Windows env handling (#22927)
* fixes #22441
* fixes #22655
* moves all env handling into the exec wrapper; this should work for everything but raw, which is consistent with non-Windows.
2017-03-23 17:48:15 -07:00
Toshio Kuratomi
2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
2017-03-23 13:35:05 -07:00
Toshio Kuratomi
589e217278 base64 functions must be given byte strings
The base64 stdlib functions require byte strings.  So we have to
transform the strings into bytes before handing to the stdlib and then
transform them back into text before handing back to ansible.

Fixes #22873
2017-03-23 12:15:02 -07:00
Peter Sprygada
cda3e001c6 fixes issue when host is defined in provider for junos (#22918)
The junos action handler was not honoring the host value in the provider
argument.  This patch will now use the provider host entry if it exists
and falls back to the inventory hostname
2017-03-23 15:05:11 -04:00
Matt Martz
0ced29c51b Don't skip action/__init__.py (#22911) 2017-03-23 13:40:02 -05:00
Adrian Likins
0bb3545333 Remove ununsed Connection.log method (#22837)
method referenced 'logger' which no longer exists.
2017-03-23 12:24:39 -04:00
Adrian Likins
dfec69b9a1 Fix bad ref to AnsibleConnectionError -> AnsibleConnectionFailure (#22838) 2017-03-23 10:43:02 -04:00
Adrian Likins
ee478c4885 make netconf.py stylish (#22839)
(flake8 stuff)

netconf.py:22:1: F401 're' imported but unused
netconf.py:23:1: F401 'socket' imported but unused
netconf.py:24:1: F401 'json' imported but unused
netconf.py:25:1: F401 'signal' imported but unused
netconf.py:31:1: F401 'ansible.module_utils.six.moves.StringIO' imported but unused
2017-03-23 10:29:24 -04:00
Brian Coca
91a385b87d etcd doc tests 2017-03-23 01:27:19 -04:00
Brian Coca
d14634ca21 uncommit the crime of extra spaces in docstring 2017-03-23 01:27:19 -04:00
Brian Coca
7839f70e36 Enable documentation in plugins
Made ansible-doc more plugin agnostic
We can have docs in lookup, callback, connectionm strategy, etc
Use first docstring and make pepizis happy
generalized module_docs to plugin_docs
documented cartesian, ssh, default, jsonfile, etc as examples
changed lack of docs to warning when listing
made smarter about bad docstrings
better blacklisting
added handling of options/config/envs/etc
move blacklist to find_plugins, only need once
2017-03-23 01:27:19 -04:00
Brian Coca
d3115ae8f3 put cache base classes in __init__ (#22809)
* put cache base classes in __init__

* added placeholder
2017-03-23 01:26:10 -04:00
Evgeni Golov
7a00f28804 synchronize: explicitly set the executable for localhost
Otherwise the executable for the destination is also used on the local
machine and this might not exist.

Fixes: #22867
2017-03-22 21:27:47 -07:00
Matt Martz
3164e8b561 E501 fixes (#22879) 2017-03-22 20:50:28 -05:00
Brian Coca
5e98be2e62 better handling of parsed
updated tests to match new result output
2017-03-22 12:03:43 -04:00
Brian Coca
9ad62a48ab prevent failure for meta: reset_connection
now plugins that don't explicitly support it give a warning.
2017-03-22 11:00:32 -04:00
Matt Martz
02f66b9369 E1 legacy pep8 fixes (#21933)
* E1 pep8 fixes

* e111 fix for rds.py
2017-03-21 21:19:40 -05:00
Ted Timmons
887456ab8e Fix stderr false return value (#22845)
* ensure exit_json does not fail from stderr=False

- do a little bit of safety-checking in exit_json to not try to .splitlines() on a boolean
- remove the stderr=boolean from uri.py, this is the only spot that uses it (at least so obviously)
- add unit tests that call exit_json. These are useless because the stderr parsing is in _execute_module and is difficult to mock; deleting these tests after the commit.

* remove added unit tests per prev commit

exit_json doesn't do the param parsing, that is buried deep inside _execute_module.
2017-03-21 18:11:11 -04:00
Brian Coca
5beb27ef5e use namespaced facts 2017-03-21 17:10:15 -04:00
Peter Sprygada
e20ed8bc0c replaces logging with display for network connection plugins (#22819)
* updates network_cli and netconf connection plugins
* updates ansible-connection
2017-03-20 20:08:02 -07:00
Peter Sprygada
ed7cace425 implements python logging for network connection plugin (#22817)
* enables logging for network_cli and paramiko
* enables logging for ansible-connection
* enabled logging for netconf connection
2017-03-20 19:26:18 -07:00
Matt Davis
ffa4f0c427 Suppress Windows Add-Type debug noise (#22722) 2017-03-16 15:44:42 -07:00
jralbert
5334814396 Cast syslogport value from getenv as int (#22664)
SysLogHandler requires an int for port, but getenv produces a string.
Cast return from getenv as int to overcome this mismatch.
2017-03-16 10:38:55 -07:00
Brian Coca
4fb8b03f05 added docstring to get_hosts_left 2017-03-16 12:15:17 -04:00
Brian Coca
6b46cc5c73 added new 'order' directive to sort hosts in play
fixes #10964
2017-03-16 12:13:29 -04:00
Brian Coca
dd8d699981 namespace facts (#18445)
* namespace facts

always namespace facts, make the polluting of 'main' conditional on config

* updated to 2.4

* Update intro_configuration.rst
2017-03-15 17:12:16 -07:00
Brian Coca
0c44959800 added new tests any and all (#22665)
* added new tests any and all

* updated code names
2017-03-15 18:09:25 -04:00
Brian Coca
7b0f765a57 made error slightly more neutral in case of 'local' 2017-03-15 15:32:30 -04:00
Matt Davis
ce56da69b2 make windows async ... async (#22624)
Fixes #22575 - issue under new exec wrapper where unconstrained handle inheritance (for stdin) caused WinRM to block on breakaway processes. Uses explicit handle inheritance to ensure that only stdin read handle gets inherited. Adds test to ensure that async is actually async.
2017-03-14 16:37:55 -07:00
Matt Davis
07674a8446 prevent winrm Add-Type debug noise from polluting stderr (#22583) 2017-03-13 14:47:06 -07:00
Ganesh Nalawade
8b6d786cff Fix junos prompt issue (#22536)
Need to handle junos shell prompt
that end's with '%'
2017-03-13 08:11:38 -04:00
Peter Sprygada
3f0f7c4f4e updates junos shared lib and action handler (#22541)
* removes cli functions from shared lib
* adds cli functions to junos_netconf module
* statically pins junos_netconf to cli transport
* all other modules use netconf transport
* adds command rpc function to junos shared
2017-03-12 08:37:45 -05:00
Peter Sprygada
1825406e1e Junos fixes (#22423)
* Fixes for junos_config errors

* Check transport settings for core Junos

* Don't pop from the same list you iterate over

* use of persistent connections are now explicitly enabled in junos

* modules must now explicitly enable persistent connections
* adds rpc support to junos_command

fixes #22166
2017-03-11 10:26:42 -06:00
Peter Sprygada
17fc6832ca fixes mapping nxapi arguments for nxos action plugin (#22491)
The nxos action plugin does not properly map task arguments when using
provider.  This patch will fix the problem and properly map the
arguments
2017-03-11 09:46:15 -06:00
Peter Sprygada
027f5955cc correctly maps task arguments for eapi transport (#22490)
The provider arguments where not correctly being mapped in the action
plugin for eapi transport.  This will now correctly map existing
arguments as well as set the correct defaults for values.

fixes #22442
2017-03-11 09:45:55 -06:00
Ricardo Carrillo Cruz
9173a7727c Fix issue on nxos modules when transport is passed via provider (#22488)
The nxos modules read the task level transport variable, thus if
the user pass it via provider the all fail with an UnboundLocalError.

Fixes #22355
2017-03-10 07:32:24 -07:00
Matt Clay
0b250016b0 Fix PEP 8 issue. 2017-03-08 10:05:19 -08:00
Brian Coca
406505bfe5 fixes cache file modules with bad path permissions
now module will fail to load and report an error (turned into warning as this should not be fatal)
fixes #13093
2017-03-08 10:58:53 -05:00
Matt Martz
1fe67f9f43 Extend SSH Retry to put_file and fetch_file (#20187)
* Move retry logic into _ssh_retry decorator, and apply to exec_command, put_file and fetch_file

* Update tests to reflect change

* Move _ssh_retry to _run, and update tests to reflect

* piped should use exec_command instead of removed _exec_command

* Rework tests to support selectors instead of select.select
2017-03-02 16:56:29 -06:00
Peter Sprygada
7b2c013e5a returns xml as a display option to junos_command (#21833)
The display option xml as accidentially removed from the display
argument.  This patch adds xml back as an option.

fixes #21823
2017-03-02 16:05:20 -06:00
Matt Davis
42ccf9ca98 tweak transport_tests to avoid logic duplication (#22158) 2017-03-01 15:54:27 -08:00
Dag Wieers
52959ebdc1 wait_for_connection: Wait for system to become reachable (#20011)
* WIP: wait_for_connection: Wait for system to be reachable

This action plugin allows to check when a system is back online and
usable by Ansible.

As an example, when doing a SysPrep and running Enable-WinRM.ps1, it
takes between 10 to 20 seconds between the WinRM TCP port to open, and
it actually being able to server Ansible requests. This time is variable
and depends on the boot process.

Current implementation is specific for Windows (WinRM) only, this will
be fixed shortly.

This fixes #19998

* Support other transport types

* Various improvements

- Fix reported typo
- Add transport_test support in accelerate plugin
- Ensure port is an integer

* Improve examples

* Small fixes

- Use correct ConfigureRemotingForAnsible.ps1 script name
- Only use win_ping when remote shell is known to be Powershell
- Add integration tests to CI framework
2017-03-01 11:00:49 -08:00
Nathaniel Case
d9d2e6deb6 Junos provider readd (#21869)
* Restore `provider` to junos_*

Fixes #21824
Fixes #21824
Fixes #21827

* Fix `confirm_timeout` related errors
* Fix glaring issues with _junos_template
2017-03-01 12:39:44 -05:00
Ricardo Carrillo Cruz
029ccf9aa0 Check right CLI context is set on iosxr action plugin (#22108)
Fixes #21994
2017-03-01 10:44:12 +00:00
Brian Coca
bfb3467d52 more set/list stuff 2017-02-28 10:44:20 -05:00
Matt Davis
8265860f07 suppress PS "unapproved verb" warning (#22018) 2017-02-27 14:29:43 -08:00
Brian Coca
b3abab1bd5 Fact proccessing fixes (#22003)
* clarify facts assignment for several corner cases

run_once/delegate_facts:
 now delegate_facts > run_once, previously run_once always published facts to all hosts in play

include_vars/delegate_to:
  now include_vars allows to delegate to a specific host

also fix task_vars exception in delegate_facts/loop as var was removed

fixes #15365

* removed unused loop_var
2017-02-27 16:22:30 -05:00
Brian Coca
b3251c9585 Exceptit (#21864)
* more centralized exception handling

* only remove when verboxse
2017-02-27 13:29:40 -05:00
Nathaniel Case
6d9fb4d012 Network action plugin misusing display.debug (#21995)
* The correct answer was in fact display.vvvv
2017-02-27 12:34:31 -05:00
Paul Arthur
9c72d478ec hashi_vault: fix token logic
The token should not be set and checked twice, especially when the
second time overrides a previously set token.
2017-02-27 09:36:33 -05:00
Peter Sprygada
0cb2019293 roll up of bug fixs for nxos_evpn_global (#21961)
* updates nxos_evpn_global module
* adds integration test cases
* adds unit test cases
2017-02-26 06:45:26 -05:00
Nathaniel Case
9b9ed59d98 Vyos config fixes (#21862)
* Fix vyos signatures to match new versions

* Fix test cases referring to 'updates' instead of 'commands'

* I think this is an artifact of `connection: network_cli`?
2017-02-25 21:14:40 -05:00
Nathaniel Case
ca21d09483 Fix a few errors (#21854) 2017-02-25 21:12:49 -05:00
Peter Sprygada
f9b108e319 fixes issue that caused eapi transport setting to get lost (#21919)
The eos action didn't properly set provider transport argument for the
module.  This patch fixes that problem
2017-02-25 15:20:32 -05:00
Dag Wieers
98934939af win_copy: Add force parameter and check-mode support (#20405)
* win_copy: Add force parameter and check-mode support

The rationale behind this is that if you're working with +3GB files,
creating the checksum takes a lot of time, which we can avoid by simply
testing if the file exists.

I also took the liberty to put the various parameters together. It
probably takes a (neglible) performance hit but makes the code a bit
easier to inspect/work with, as its closer to all other windows modules.

On a normal run, the action plugin does a local checksum of the source
and a remote checksum of the destination. And afterwards, the module
will do another remote checksum of the copied source, a remote checksum
of the original destination, and another remote checksum of the copied
destination.

On a very huge file (think 4GB) that means 5x reading the complete file
(if you have a large cache you may get away with it, otherwise you're
doomed !).

This patch will ensure with `force: no` that not checksums are being
performed.

* Moving presence check before remote checksum

* Adapted to wishes

* Even more performance improvements
2017-02-24 18:10:09 -08:00
Matt Martz
305c88700d PEP8 E712 fixes (#21916) 2017-02-24 16:49:43 -06:00
Adrian Likins
8830cde28d Fix 'task name is not templated in retry callback' (add task_name property to TaskResult) (#21214)
Fix 'task name is not templated in retry callback'

Add a task_name property to TaskResult that knows to
check in TaskResult._task_fields.

Add integration test for v2_retry_runner callback

Fixes #18236
2017-02-24 12:33:24 -05:00
Dag Wieers
cfb1f72d1c Improve winrm import error message (#20267)
The requests python module is needed, however it is not a dependency of
the python-winrm package. The python-winrm package does require
python-requests_ntlm, which does not seem to pull python-requests.

So for the time being (until Red Hat fixes their package) give a more
informative error message.
2017-02-23 23:42:23 -08:00
Peter Sprygada
ab0f992b3a fixes import statement in nxos_facts module (#21834)
trying to import from the wrong shared lib, this patch changes
network_common to connection.  Also fixes a bug found in the nxos
action plugin to detect transport

fixes #21829
2017-02-23 13:00:34 +00:00
Peter Sprygada
0e7ebf6391 fixes issue with setting transport to nxapi (#21811)
The nxos action handler did not map the nxapi value to the provider
argument properly.  There as an additional fix in the nxos shared lib
to properly detect nxapi being set
2017-02-22 22:07:33 -05:00
jctanner
8c521655e1 Fix traceback on indexing empty list (#21784)
* Fix traceback on indexing empty list
* Make it more 'idiomatic'
* Forgot the index
2017-02-22 16:14:39 -05:00
Peter Sprygada
f8157d579a fixes error where a transport is not set for nxos (#21790)
Will now make sure transport is set to `cli`
2017-02-22 12:47:21 -05:00
Brian Coca
93ac278f4c added 'extensions' options to include_vars
This allows users to override default 'extensions read' when using the dir option
2017-02-22 10:31:22 -05:00
Toshio Kuratomi
45251f910c Make BaseFileCache into an abstractbaseclass so it's a proper interface
Push the opening and closing of files into the _load and _dump methods
so that we don't invoke the slow codec machinery without reason.
2017-02-22 10:27:29 -05:00
Toshio Kuratomi
c033e5111f Graceful error message for abstract base classes in PluginLoader 2017-02-22 10:27:29 -05:00
Brian Coca
374af06cbf added pickle and yaml cache plugins
added new base class for file based cache plugins as 99% of code was common
now also catches unexpected decoding exceptions
allows per module file modes and encoding
moved jsonfile code to base
2017-02-22 10:27:29 -05:00
Toshio Kuratomi
2dde7f7768 Fix a case where we mixed text and bytes in the local connection plugin
Fixes #20710
2017-02-21 14:19:08 -08:00
James Cammarata
3d65482927 Stash post-validated fields of the task in the TaskResult
This allows us to have a snapshot of the fields, which we can restore
on the pre-fork side so as to avoid having to re-template fields.
2017-02-21 14:35:13 -06:00
Peter Sprygada
6e9244a9e1 adds more logging output to network_cli and ansible-connection (#21716) 2017-02-21 09:21:41 -05:00
Peter Sprygada
2567e8157f fixes bug where handle_prompts failed if list of prompts provided (#21711)
This will now automatically convert the prompts to a list and iterate
over the regexp looking for a prompt match before supplying the answer.
2017-02-21 08:27:33 -05:00
Peter Sprygada
e4a2c804be partial revert of changes introduced in d5f7a0181b (#21688) 2017-02-20 15:37:14 -05:00
Marc Abramowitz
e2e4a69425 hashi_vault: Get token from env var or file
This allows getting the Vault token from the `VAULT_TOKEN` env var or
from the file `$HOME/.vault-token`, as both of these are understood by
the Vault CLI and are a common place to put Vault tokens. This allows
avoiding hard-coding a Vault token into playbooks or having to include
lookups.

`HOME/.vault-token` is nice because a user can authenticate with the CLI
using `vault auth` and then the token will be stored in
`$HOME/.vault-token`. If we read this file, then we allow someone to do
`vault auth` "out of band" to set up Vault access.
2017-02-20 10:10:22 -05:00
René Moser
576ff0728d service: deprecate state=running (#21499)
* service: deprecate state=running

* deprecated with version 2.7
2017-02-20 09:34:52 -05:00
Peter Sprygada
bd036c15e0 minor fixes to junos action (#21643)
* adds instrumentation messages
* checks cli context is not in config mode
2017-02-19 18:04:00 -05:00
Peter Sprygada
ce9180b3a1 roll up of vyos action fixes (#21642)
* check cli context is not in config mode
* add display messages for more details
2017-02-19 17:37:16 -05:00
Peter Sprygada
8472ed640b fixes minor bugs in nxos action (#21641)
* checks cli context is correct
* adds display messages
2017-02-19 17:13:14 -05:00
Peter Sprygada
d5f7a0181b roll up of eos fixes (#21629)
* fixes error where eos would close the cli shell
* fixes network_cli connection plugin to check before calling open_shell()
* fixes json commands being sent over eapi
2017-02-19 08:46:14 -05:00
Peter Sprygada
2f10bdf0c7 roll up of fixes in junos action plugin (#21624)
* calls open_shell() or open_session() depending on connection type
* closes shell after module completion
* adds open_session() to netconf
2017-02-18 17:20:03 -05:00
Peter Sprygada
920f9f4815 fixes nxos nxapi implementation (#21615)
* correctly maps play_context to nxapi values
* fixes bug in nxos_nxapi module detecting nxapi feature
* updates nxos shared lib provider values
* fixes missing ssh_keyfile in nxos shared lib
2017-02-18 11:20:26 -05:00
Peter Sprygada
c875393916 roll up of fixes for vyos base (#21616)
* adds ssh_keyfile to vyos
* fixes play_context mapping to provider
* adds missing network_os to play_context
* executes close_shell() after module completes
2017-02-18 09:29:17 -05:00
Peter Sprygada
e6dfbf63f6 fixes eos action mapping of play_context values (#21614)
* adds mapping for ssh_keyfile
* adds correct mappings for eapi
* fixes missing no_log value for provider arg
2017-02-18 09:29:06 -05:00
Brian Coca
ee17b91455 Revert "add systemd-nspawn connection driver"
This reverts commit 3597ca082b.
2017-02-17 16:36:46 -05:00
Brian Coca
e84713c76c Revert "Remove the --nspawn-extra-args cli option"
This reverts commit b8125ac1a6.
2017-02-17 16:35:47 -05:00
Brian Coca
2d2874fb3c Revert "Add safety checks to nspawn connection plugin"
This reverts commit 60bb677154.

this connection cannot work as is, systemd-nspawn creates instance on each command
2017-02-17 16:35:47 -05:00
Toshio Kuratomi
275be702a8 Switch from using inspect to globals() 2017-02-17 12:57:26 -08:00
Peter Sprygada
8716a5bc67 maps ios provider ssh_keyfile to play_context (#21597)
This allows the key file to be specified in the playbook for the ios
module provider argument
2017-02-17 18:56:00 +00:00
Thomas Szymanski
60bb677154 Add safety checks to nspawn connection plugin
This patch adds some checks on the path that is accessed as a container,
making sure it looks like one. It implements the connection method and
add adaptations to the modern way of writing connections for Ansible.
It also rewords docs and vars to use the nspawn terminology instead of
chroot.
2017-02-17 12:39:48 -05:00
Thomas Szymanski
b8125ac1a6 Remove the --nspawn-extra-args cli option
Providing extra arguments to pass to systemd-nspawn is still possible
through the `nspawn_connection` of the `ansible.cfg` file.
2017-02-17 12:39:48 -05:00
Lars Kellogg-Stedman
3597ca082b add systemd-nspawn connection driver
This commit adds a connection driver built on top of systemd-nspawn.
This is similar to the existing `chroot` driver, except that nspawn
offers a variety of additional services. For example, it takes care of
automatically mounting `/proc` and `/sys` inside the chroot environment,
which will make a variety of tools work correctly that would otherwise
fail.

You can take advantage of other system-nspawn features to perform more
complicated tasks.  For example, on my x86_64 system I have a Raspberry
Pi disk image mounted on `/rpi`.  I can't use `chroot` with this because
the binaries contained in the image are for the wrong architecture.
However, I can use the systemd-nspawn `--bind` option to automatically
insert the appropriate qemu-arm binary into the container using an
inventory file like this:

    pi ansible_host=/rpi ansible_nspawn_extra_args='--bind /usr/bin/qemu-arm --bind /lib64'

See http://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
for more information about systemd-nspawn itself.
2017-02-17 12:39:48 -05:00
Peter Sprygada
a6cecef6bc clean up of terminal plugins (#21549)
* removes unneeded supports_multiplexing var
* refactors terminal_prompts_re to terminal_stdout_re
* refactors terminal_errors_re to terminal_stderr_re
* updates network_cli unit test cases
2017-02-17 10:00:23 -05:00
Ricardo Carrillo Cruz
c5452eef6f Removes default kwarg in iosxr_argument_spec for timeout (#21569) 2017-02-17 15:59:24 +01:00
Peter Sprygada
4cbbed0b37 fixes issue with prompt detection in network_cli (#21574)
The network_cli plugin would return immediately if an error was
detected.  This patch will force the connection plugin to still try to
detect the current prompt even if an error is found.
2017-02-17 14:13:26 +00:00
Peter Sprygada
b0abbb5f8b removes the default kwarg in ios_argument_spec for timeout (#21552)
Uses the configured timeout setting instead of the arg_spec

fixes #21520
2017-02-17 13:12:12 +01:00
Matt Davis
8527013fbe Complete rewrite of Windows exec wrapper (#21510)
* supports pipelining for faster execution
* supports become (runas), creates interactive subsession under WinRM batch logon
* supports usage of arbitrary module_utils files
* modular exec wrapper payload supports easier extension
* integrates async wrapper behavior for pipelined/become'd async
* module_utils are loaded as true Powershell modules, no more runtime modifications to module code
2017-02-17 00:09:56 -08:00
James Cammarata
7bf56ceee3 Relocate creation of Templar in process_pending_results
Moving it to after the blocks where per-item results are calculated,
as it's not used there and causes quite a performance hit being there.

Fixes #21340
2017-02-17 00:26:09 -06:00
Peter Sprygada
c9f6a2b740 fixes issue with cli shell left open (#21548)
The nxos action plugin will now close the shell connection once the
module has completely run
2017-02-16 22:11:32 -05:00
Peter Sprygada
20c5a1adc1 verifies cli context for iosxr (#21550)
Checks cli context and exits config mode if needed
2017-02-16 22:11:21 -05:00
Peter Sprygada
9d4a3599b8 bug fixes and updates for eos connections (#21534)
* refactors supports_sessions to a property
* exposes supports_sessions as a toplevel function
* adds open_shell() to network_cli
* implements open_shell() in eos action plugin
2017-02-16 20:26:48 -05:00
Peter Sprygada
a01288859d check ios cli context and exits config mode if there (#21544) 2017-02-16 20:26:29 -05:00
Toshio Kuratomi
eeaec56ed5 Use isinstance instead of type() comparisons. isinstance is more robust 2017-02-16 16:34:43 -08:00
Peter Sprygada
02d2b753db refactors junos modules to support persistent socket connections (#21365)
* updates junos_netconf module
* updates junos_command module
* updates junos_config module
* updates _junos_template module
* adds junos_rpc module
* adds junos_user module
2017-02-16 10:53:03 -05:00
Brian Coca
cc0bb54d2c the return of reset_connection
allows user to force persistent connection to close, needed for when
you want to benefit from changes applied to the current play but persistent connections
prevent them from being realized.
2017-02-16 10:49:57 -05:00
Peter Sprygada
34e6cc788f check cli context to be sure out of config mode in ios (#21493)
This change will now check the cli context after a module runs and if
the cli is still in config mode it will exit config mode.  Also fixes a
minor issue with converting list of commands to a dict

fixes #21481
2017-02-16 12:54:33 +00:00
James
87646595e3 Remove the end of the task_data.path which causes #21461 2017-02-15 17:35:16 -08:00
Michael Scherer
6fae1d2bbf Make password lookup treat /dev/null as a special case
I have from time to time a need of random password without
wanting to write them down (one example is mailman list creation,
that requires a password to be given to be sent to the list owner).

But using /dev/null do not return null, but the empty string, which
doesn't generate a password at all and so do not achieve my use case.
2017-02-15 12:15:09 -08:00