Implement tag and skip_tag handling in the CLI() class. Change tag and
skip_tag command line options to be accepted multiple times on the CLI
and add them together rather than overwrite.
* Make it configurable whether to merge or overwrite multiple --tags arguments
* Make the base CLI class an abstractbaseclass so we can implement
functionality in parse() but still make subclasses implement it.
* Deprecate the overwrite feature of --tags with a message that the
default will change in 2.4 and go away in 2.5.
* Add documentation for merge_multiple_cli_flags
* Fix galaxy search so its tags argument does not conflict with generic tags
* Unit tests and more integration tests for tags
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing. So once created it we had two sets of functions that did
the same things but had different implementations. To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
* dynamic role_include
* more fixes for dynamic include roles
* set play yfrom iterator when dynamic
* changes from jimi-c
* avoid modules that break ad hoc
TODO: should really be a config
* attempt #11 to role_include
* fixes from jimi-c
* do not override load_data, move all to load
* removed debugging
* implemented tasks_from parameter, must break cache
* fixed issue with cache and tasks_from
* make resolution of from_tasks prioritize literal
* avoid role dependency dedupe when include_role
* fixed role deps and handlers are now loaded
* simplified code, enabled k=v parsing
used example from jimi-c
* load role defaults for task when include_role
* fixed issue with from_Tasks overriding all subdirs
* corrected priority order of main candidates
* made tasks_from a more generic interface to roles
* fix block inheritance and handler order
* allow vars: clause into included role
* pull vars already processed vs from raw data
* fix from jimi-c blocks i broke
* added back append for dynamic includes
* only allow for basename in from parameter
* fix for docs when no default
* fixed notes
* added include_role to changelog
Make some python3 fixes to make the unittests pass:
* galaxy imports
* dictionary iteration in role requirements
* swap_stdout helper for unittests
* Normalize to text string in a facts.py function
Fixes for these are either rewriting to get rid of the need for the
functions or using six.moves to get equivalent functions for both
python2 and python3
If someone run:
ansible all -m file state=present
The error message is "Missing target hosts" which is misleading, since
the target hosts is here, the problem is the missing '-a'.
The output of 'ansible-galaxy info' was formatting the
'galaxy_info' key with one char per line.
Previously, when building the output string, items in
role_info that had a dict for value, the label for
it's key ('galaxy_info' for ex) was being added to
the text list in addition to being appended. Only
the append is needed.
Also added a unit test in test/units/cli/test_galaxy.py,
but skip it on py3 until galaxy is py3 compatible.
fixes#15177
* Make ziploader's ansible and ansible.module_utils libraries into
namespace packages.
* Move __version__ and __author__ from ansible/__init__ to
ansible/release.py. This is because namespace packages only load one
__init__.py. If that is not the __init__.py with the author and
version info then those won't be available.
* In ziplaoder, move the version ito ANSIBLE_CONSTANTS.
* Change PluginLoader to properly construct the path to the plugins even
when namespace packages are present.
rm _del_ as it might leak memory
renamed to tmp file cleanup
added exception handling when traversing file list, even if one fails try rest
added cleanup to finally to ensure removal in most cases
* Can be configured in the ansible.cfg for tasks/handlers individually
* If an included filename contains no vars or loops, it will be expanded
in-place as if it were marked as static
porting @dominis 's ansible-shell tool from 1.9 and integrating it into ansible
added verbosity control
made more resilitent to several errors
added highlight color, to configurable colors
more resilient on exception and interruptions
prompt coloring, goes red and changes to # when using become = true and root
become setting is now explicit and not a toggle
- moved to base cli class to handle centrally and duplicate less code
- now avoids duplication and reiteration of signal handler by reassigning it
- left note on how to do non-graceful in case we add in future
as I won't remember everything i did here and don't want to 'relearn' it.
- adhoc now terminates gracefully
- avoid race condition on terminations by ignoring errors if
worker might have been reaped between checking if active and termination call
- ansible-playbook now properly exits on sigint/term
- adhoc and playbook now give exceptions that we should not normally capture
and rely on top level finally to reap children
- handle systemexit breaks in workers
- added debug to see at which frame we exit
partial fix for #14346
The module docs and vault changes solve issues where tracebacks can
happen. The galaxy changes are mostly refactoring to be more pythonic
with a small chance that a unicode traceback could have occurred there
without the changes. The change in __init__.py when we actually call
the pager makes things more robust but could hide places where we had
bytes coming in already so I didn't want to change that without auditing
where the text was coming from.
Fixes#14178
Note that the fix for display normalizing to unicode is correct but the
fix for pathnames is probably not. Changing pathnames to unicode type
means that we will handle utf8 pathnames fine but pathnames can be any
sequence of bytes that do not contain null. We do not handle sequences
of bytes that are not valid utf8 here. To do that we need to revamp the
handling of basedir and paths to transform to bytes instead of unicode.
Didn't want to do that in 2.0.x as it will potentially introduce other
bugs as we find all the places that we combine basedir with other path
elements. Since no one has raised that as an issue thus far so it's not
something we need to handle yet. But it's something to keep in mind for
the future.
To test utf8 handling, create a utf8 directory and run a playbook from
within there.
To test non-utf8 handling (currently doesn't work as stated above), create
a directory with non-utf8 chars an run a playbook from there. In bash,
create that directory like this: mkdir $'\377'
Fixes#13937
* sudo was not working, now it supports full become
* now default checkout dir works, not only when specifying
* paths for checkout dir get expanded
* fixed limit options for playbook
* added verbose and debug info
since all the --ask pass options end up triggering the same code
and are functionally equivalent, ignore them when it comes to checking
privilege escalation conflicts. This allows using -K when --become-method=su
and so on.
* remove requirement for host patterns, use the defaults
* require destination directory (None in cwd is not a good default)
* fixed usage messages
* updated default inventory to use , and not deprecated :
better error reporting on fetching errors
use scm if it exists over src
unified functions in requirements
simplified logic
added verbose to tests
cleanup code refs, unused options and dead code
moved get_opt to base class
fixes#11920fixes#12612fixes#10454
Now we have the following ways to set additional arguments:
1. [ssh_connection]ssh_args in ansible.cfg: global setting, prepended to
every command line for ssh/scp/sftp. Overrides default ControlPersist
settings.
2. ansible_ssh_common_args inventory variable. Appended to every command
line for ssh/scp/sftp. Used in addition to ssh_args, if set above, or
the default settings.
3. ansible_{sftp,scp,ssh}_extra_args inventory variables. Appended to
every command line for the relevant binary only. Used in addition to
#1 and #2, if set above, or the default settings.
3. Using the --ssh-common-args or --{sftp,scp,ssh}-extra-args command
line options (which are overriden by #2 and #3 above).
This preserves backwards compatibility (for ssh_args in ansible.cfg),
but also permits global settings (e.g. ProxyCommand via _common_args) or
ssh-specific options (e.g. -R via ssh_extra_args).
Fixes#12576
CLI already provides a pager() method that feeds $PAGER on stdin, so we
just feed that the plaintext from the vault file. We can also eliminate
the redundant and now-unused shell_pager_command method in VaultEditor.
Required some rewiring in inventory code to make sure we're using
the DataLoader class for some data file operations, which makes mocking
them much easier.
Also identified two corner cases not currently handled by the code, related
to inventory variable sources and which one "wins". Also noticed we weren't
properly merging variables from multiple group/host_var file locations
(inventory directory vs. playbook directory locations) so fixed as well.
commit 9921bb9d2002e136c030ff337c14f8b7eab0fc72
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Mon Aug 10 20:19:44 2015 +0530
Document --ssh-extra-args command-line option
commit 8b25595e7b1cc3658803d0821fbf498c18ee608a
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Thu Aug 13 13:24:57 2015 +0530
Don't disable GSSAPI/Pubkey authentication when using --ask-pass
This commit is based on a bug report and PR by kolbyjack (#6846) which
was subsequently closed and rebased as #11690. The original problem was:
«The password on the delegated host is different from the one I
provided on the command line, so it had to use the pubkey, and the
main host doesn't have a pubkey on it yet, so it had to use the
password.»
(This commit is revised and included here because #11690 would conflict
with the changes in #11908 otherwise.)
Closes#11690
commit 119d0323892c65e8169ae57e42bbe8e3517551a3
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Thu Aug 13 11:16:42 2015 +0530
Be more explicit about why SSH arguments are added
This adds vvvvv log messages that spell out in detail where each SSH
command-line argument is obtained from.
Unfortunately, we can't be sure if, say, self._play_context.remote_user
is obtained from ANSIBLE_REMOTE_USER in the environment, remote_user in
ansible.cfg, -u on the command line, or an ansible_ssh_user setting in
the inventory or on a task or play. In some cases, e.g. timeout, we
can't even be sure if it was set by the user or just a default.
Nevertheless, on the theory that at five v's you can use all the hints
available, I've mentioned the possible sources in the log messages.
Note that this caveat applies only to the arguments that ssh.py adds by
itself. In the case of ssh_args and ssh_extra_args, we know where they
are from, and say so, though we can't say WHERE in the inventory they
may be set (e.g. in host_vars or group_vars etc.).
commit b605c285baf505f75f0b7d73cb76b00d4723d02e
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Tue Aug 11 15:19:43 2015 +0530
Add a FAQ entry about ansible_ssh_extra_args
commit 49f8edd035cd28dd1cf8945f44ec3d55212910bd
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Mon Aug 10 20:48:50 2015 +0530
Allow ansible_ssh_args to be set as an inventory variable
Before this change, ssh_args could be set only in the [ssh_connection]
section of ansible.cfg, and was applied to all hosts. Now it's possible
to set ansible_ssh_args as an inventory variable (directly, or through
group_vars or host_vars) to selectively override the global setting.
Note that the default ControlPath settings are applied only if ssh_args
is not set, and this is true of ansible_ssh_args as well. So if you want
to override ssh_args but continue to set ControlPath, you'll need to
repeat the appropriate options when setting ansible_ssh_args.
(If you only need to add options to the default ssh_args, you may be
able to use the ansible_ssh_extra_args inventory variable instead.)
commit 37c1a5b6794cee29a7809ad056a86365a2c0f886
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Mon Aug 10 19:42:30 2015 +0530
Allow overriding ansible_ssh_extra_args on the command-line
This patch makes it possible to do:
ansible somehost -m setup \
--ssh-extra-args '-o ProxyCommand="ssh -W %h:%p -q user@bouncer.example.com"'
This overrides the inventory setting, if any, of ansible_ssh_extra_args.
Based on a patch originally by @Richard2ndQuadrant.
commit b023ace8a8a7ce6800e29129a27ebe8bf6bd38e0
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Mon Aug 10 19:06:19 2015 +0530
Add an ansible_ssh_extra_args inventory variable
This can be used to configure a per-host or per-group ProxyCommand to
connect to hosts through a jumphost, e.g.:
inventory:
[gatewayed]
foo ansible_ssh_host=192.0.2.1
group_vars/gatewayed.yml:
ansible_ssh_extra_args: '-o ProxyCommand="ssh -W %h:%p -q bounceuser@gateway.example.com"'
Note that this variable is used in addition to any ssh_args configured
in the [ssh_connection] section of ansible.cfg (so you don't need to
repeat the ControlPath settings in ansible_ssh_extra_args).
Replace .iteritems() with six.iteritems() everywhere except in
module_utils (because there's no 'six' on the remote host). And except
in lib/ansible/galaxy/data/metadata_template.j2, because I'm not sure
six is available there.
Now we issue a "Reading … from stdin" prompt if our input isatty(), as
gpg does. We also suppress the "x successful" confirmation message at
the end if we're part of a pipeline.
(The latter requires that we not close sys.stdout in VaultEditor, and
for symmetry we do the same for sys.stdin, though it doesn't matter in
that case.)
This allows the following invocations:
# Interactive use, like gpg
ansible-vault encrypt --output x
# Non-interactive, for scripting
echo plaintext|ansible-vault encrypt --output x
# Separate input and output files
ansible-vault encrypt input.yml --output output.yml
# Existing usage (in-place encryption) unchanged
ansible-vault encrypt inout.yml
…and the analogous cases for ansible-vault decrypt as well.
In all cases, the input and output files can be '-' to read from stdin
or write to stdout. This permits sensitive data to be encrypted and
decrypted without ever hitting disk.
Now we don't have to recreate VaultEditor objects for each file, and so
on. It also paves the way towards specifying separate input and output
files later.
It's unused and unnecessary; VaultLib can decide for itself what cipher
to use when encrypting. There's no need (and no provision) for the user
to override the cipher via options, so there's no need for code to see
if that has been done either.
The --new-vault-password-file option works the same as
--vault-password-file but applies only to rekeying (when
--vault-password-file sets the old password). Also update the manpage
to document these options more fully.
Apart from ansible-vault create, every vault subcommand is happy to deal
with multiple filenames, so we can check that there's at least one, and
make create check separately that there aren't any extra.
Without at least one worker process, things break:
Traceback (most recent call last):
File "/home/ams/extern/ansible/ansible/lib/ansible/executor/process/result.py", line 103, in run
result = self._read_worker_result()
File "/home/ams/extern/ansible/ansible/lib/ansible/executor/process/result.py", line 69, in _read_worker_result
(worker_prc, main_q, rslt_q) = self._workers[self._cur_worker]
IndexError: list index out of range
