* [WIP] become plugins
Move from hardcoded method to plugins for ease of use, expansion and overrides
- load into connection as it is going to be the main consumer
- play_context will also use to keep backwards compat API
- ensure shell is used to construct commands when needed
- migrate settings remove from base config in favor of plugin specific configs
- cleanup ansible-doc
- add become plugin docs
- remove deprecated sudo/su code and keywords
- adjust become options for cli
- set plugin options from context
- ensure config defs are avaialbe before instance
- refactored getting the shell plugin, fixed tests
- changed into regex as they were string matching, which does not work with random string generation
- explicitly set flags for play context tests
- moved plugin loading up front
- now loads for basedir also
- allow pyc/o for non m modules
- fixes to tests and some plugins
- migrate to play objects fro play_context
- simiplify gathering
- added utf8 headers
- moved option setting
- add fail msg to dzdo
- use tuple for multiple options on fail/missing
- fix relative plugin paths
- shift from play context to play
- all tasks already inherit this from play directly
- remove obsolete 'set play'
- correct environment handling
- add wrap_exe option to pfexec
- fix runas to noop
- fixed setting play context
- added password configs
- removed required false
- remove from doc building till they are ready
future development:
- deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems
* cleanup
remove callers to removed func
removed --sudo cli doc refs
remove runas become_exe
ensure keyerorr on plugin
also fix backwards compat, missing method is attributeerror, not ansible error
get remote_user consistently
ignore missing system_tmpdirs on plugin load
correct config precedence
add deprecation
fix networking imports
backwards compat for plugins using BECOME_METHODS
* Port become_plugins to context.CLIARGS
This is a work in progress:
* Stop passing options around everywhere as we can use context.CLIARGS
instead
* Refactor make_become_commands as asked for by alikins
* Typo in comment fix
* Stop loading values from the cli in more than one place
Both play and play_context were saving default values from the cli
arguments directly. This changes things so that the default values are
loaded into the play and then play_context takes them from there.
* Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH
As alikins said, all other plugin paths are named
DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that
should be done all at one time rather than piecemeal.
* One to throw away
This is a set of hacks to get setting FieldAttribute defaults to command
line args to work. It's not fully done yet.
After talking it over with sivel and jimi-c this should be done by
fixing FieldAttributeBase and _get_parent_attribute() calls to do the
right thing when there is a non-None default.
What we want to be able to do ideally is something like this:
class Base(FieldAttributeBase):
_check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check'])
class Play(Base):
# lambda so that we have a chance to parse the command line args
# before we get here. In the future we might be able to restructure
# this so that the cli parsing code runs before these classes are
# defined.
class Task(Base):
pass
And still have a playbook like this function:
---
- hosts:
tasks:
- command: whoami
check_mode: True
(The check_mode test that is added as a separate commit in this PR will
let you test variations on this case).
There's a few separate reasons that the code doesn't let us do this or
a non-ugly workaround for this as written right now. The fix that
jimi-c, sivel, and I talked about may let us do this or it may still
require a workaround (but less ugly) (having one class that has the
FieldAttributes with default values and one class that inherits from
that but just overrides the FieldAttributes which now have defaults)
* Revert "One to throw away"
This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064.
* Set FieldAttr defaults directly from CLIARGS
* Remove dead code
* Move timeout directly to PlayContext, it's never needed on Play
* just for backwards compat, add a static version of BECOME_METHODS to constants
* Make the become attr on the connection public, since it's used outside of the connection
* Logic fix
* Nuke connection testing if it supports specific become methods
* Remove unused vars
* Address rebase issues
* Fix path encoding issue
* Remove unused import
* Various cleanups
* Restore network_cli check in _low_level_execute_command
* type improvements for cliargs_deferred_get and swap shallowcopy to default to False
* minor cleanups
* Allow the su plugin to work, since it doesn't define a prompt the same way
* Fix up ksu become plugin
* Only set prompt if build_become_command was called
* Add helper to assist connection plugins in knowing they need to wait for a prompt
* Fix tests and code expectations
* Doc updates
* Various additional minor cleanups
* Make doas functional
* Don't change connection signature, load become plugin from TaskExecutor
* Remove unused imports
* Add comment about setting the become plugin on the playcontext
* Fix up tests for recent changes
* Support 'Password:' natively for the doas plugin
* Make default prompts raw
* wording cleanups. ci_complete
* Remove unrelated changes
* Address spelling mistake
* Restore removed test, and udpate to use new functionality
* Add changelog fragment
* Don't hard fail in set_attributes_from_cli on missing CLI keys
* Remove unrelated change to loader
* Remove internal deprecated FieldAttributes now
* Emit deprecation warnings now
* remove bare var handling in conditionals
this makes top level and multilevel vars (dicts keys) behave the same
it will require adding |bool for 'string comparissons' in indirect templates
- added new tests to ensure uniform handling
- switched to 'is' testing for status
- changed warning to 'conditional' as 'when:' is not only place it gets triggered
* updated to include toggle and deprecation
* fix deprecated
* updated tests to handle toggle
* fixed typo and added note about the future
* promote doc_fragments into actual plugins
change tests hardcoded path to doc fragments
avoid sanity in fragments
avoid improper testing of doc_fragments
also change runner paths
fix botmeta
updated comment for fragments
updated docs
* Increase persistent command_timeout default value
* Increase command_timeout default value from 10 to 30 sec
to reduce frequent timeout issue for network connection
types (netconf/network_cli/httpapi/napalm)
* Fix review comments
* Fix encoding issues with file paths.
Discovered while testing with ANSIBLE_CONFIG env var set to a path
that contained unicode characters while LC_ALL=C.
* Fix unit tests.
* Fix another path encoding issue.
* inventory plugins: try auto before ini
auto plugin should run before ini to avoid ini being able to parse
some plugin configuration YAML files successfully.
* Update comment
comment was added by 2ffe3c42bb but
related code was later removed by
506e6a0b2d.
* First pass at a toml inventory
* Make EXAMPLES yaml
* Remove unnecessary comment
* Small formatting changes
* Add ansible-inventory option to list as TOML
* TOML inventory improvements, to allow a more simple inventory, specifically related to children
* changelog
* Simplify logic
* Dedupe _expand_hostpattern, making it available to all inventory plugins
* Don't make the TOML inventory dependent on the YAML inventory
* Quote IP address values
* Add more TOML examples
* Further cleanups
* Enable the toml inventory to run by default
* Create toml specific dumper
* 2.8
* Clean up imports
* No toml pygments lexer
* Don't raise an exception early when toml isn't present, and move toml to the end, since it requires an external dep
* Require toml>=0.10.0
* Further clean up of empty data
* Don't require toml>=0.10.0, but prefer it, add code for fallback in older versions
* Ensure we actually pass an encoder to toml.dumps
* Simplify recursive data converter
* Appease tests, since we haven't limited controller testing to 2.7+
* Update docstring for convert_yaml_objects_to_native
* remove outdated catching of AttributeError
* We don't need to catch ImportError when import ansible.plugins.inventory.toml
* Add note about what self.dump_funcs.update is doing
* Address some things
* A little extra comment
* Fix toml availability check
* Don't create an intermediate list
* Require toml file extension
* Add metadata
* Remove TOML docs from intro_inventory to prevent people from getting the wrong idea
* It's in defaults, remove note
* core supported, indicate very clearly that this is preview status
* Removed deprecated ANSIBLE_HOSTS
* Bump sudo/su configs to match deprecation version for cli and playbook args
* Bump include configs to match deprecation version for 'include'
* Ensure that the value of PLUGIN_FILTERS_CFG is treated as type=path, and that we use the standard section of 'defaults' instead of 'default'
* deprecate the default section
* Don't add version_added for the corrected section
* gather_subset is a list of strings
When gather_subset is an integer, a message pointing out the problem, current tb error
is replaced by this one:
ERROR! the field 'gather_subset' should be a list of (<class 'str'>,), but the item '42' is a <class 'int'>
...
* gathering_facts test: ensure smart gathering is on
Extends `module_defaults` by adding a prefix to defaults `group/` which denotes a builtin list of modules. Initial groups are: `group/aws`, `group/azure`, and `group/gcp`
Fixes#40650Fixes#40245Fixes#41541
* Refactor netconf_config module as per proposal #104
* Update netconf_config module metadata to core network supported
* Refactor local connection to use persistent connection framework
for backward compatibility
* Update netconf connection plugin configuration varaibles (Fixes#40245)
* Add support for optional lock feature to Fixes#41541
* Add integration test for netconf_config module
* Documentation update
* Move deprecated options in netconf_config module
Only print warning when ansible.cfg is actually skipped
* Also add unittests for the find_ini_config_file function
* Add documentation on world writable current working directory
config files can no longer be loaded from a world writable current
working directory but the end user is allowed to specify that
explicitly. Give appropriate warnings and information on how.
Fixes#42388
If there were fatal bugs in this portion of config, they would never be displayed
because config would fail to load and then every^U (Exageration... only
half of every) other part of the code which depended on config to be
loaded would fail before we ever got around to a section of code that
would process UNABLE.
Remove the try except from here so that we are able to debug this code
On Python3, these would be text strings already. On Python2, we need to
convert them from bytes.
Use a new helper function py3compat to do this.
Fixes#43207
Fixes#37262Fixes#36284
* Updates options in netconf connection to enable
bastion/jump host setting using configuration/enviornment
varaibles.
* Update troubleshooting docs from using bastion host with netconf
connection
* pip tests: remove trailing spaces
* pip tests: use Jinja tests
* fixup! pip tests: remove trailing spaces
* pip tests: use 'command' instead of 'shell' module
* pip tests: remove unused variable
* pip tests: use a package with fewer dependencies
sampleproject has one dependency: 'peppercorn' and peppercorn doesn't
have any dependency.
* pip tests: check that 'name' param handles list
* pip: squash package parameters
Note that squashing will be removed in 2.11, new code should directly
use a list with the 'name' parameter.
When we read files from disk we always want to read them as bytes and
then convert them to text ourselves. This gives us greater control over
what encodings are used, what to do in case of errors decoding the bytes
to text, and better resilience against problems on both Python 2 and
Python 3.
If we left it up to Python to do this, on Python2, this could mean
that config values end up as bytes (leading to tracebacks elsewhere in
the code). In Python3, it could traceback if the user's locale did not
match with the encoding of the ini file or config files could be decoded
as the user's locale encoding instead of as utf-8.
In the process of building up the inventory by parsing each inventory
source with each available inventory plugin, there are three kinds of
possible errors (listed in order from earliest to latest):
1. One source could not be parsed by a particular plugin.
2. One source could not be parsed by any available plugin.
3. ALL sources could not be parsed by any available plugin.
The errors in (1) are a part of normal operation, e.g., the script
plugin is expected to fail to parse an ini-format source, and we will
ignore that error and try the next plugin. There is currently no way to
control this, and no known compelling use-case for a setting to control
it. This commit does not make any changes here.
We implement "any_unparsed_is_failed" to handle (2) above. If enabled,
this requires that every available source be parsed validly by at least
one plugin. In an inventory comprising a static hosts file and ec2.py,
this setting will cause a fatal error if ec2.py fails (a situation that
attracted only a warning earlier).
We clarify that the existing "unparsed_is_failed=true" setting causes a
fatal error only in (3) above, i.e., if NO inventory source could be
parsed. In other words, if there is ANY valid source in the inventory
(e.g., an ini-format static file), no combination of errors and the
setting will cause a fatal error.
If you want to execute your playbooks when your inventory is…
(a) complete, use "any_unparsed_is_failed=true".
(b) not empty, use "unparsed_is_failed=true".
The "unparsed_is_failed" setting should be renamed to
"all_unparsed_is_failed", but this commit does not do so.
Fixes#40512Fixes#40996
