Previously, includes had to receive variables via a special 'vars'
field. With this patch, the include syntax becomes a more natural
datastructure without special fields and is more akin to the way
role includes/dependencies work.
Tested with the following playbook:
---
- hosts: localhost
connection: local
tasks:
- { include: inc1.yml, a: 1 }
- include: inc2.yml
b: 2
- include: inc3.yml
with_items:
- x
- y
- z
Fixes#3481
This allows an included path to be relative to a directory, if the
basedir of the original path is a symlink. In that case, the path
is normalized and the file is searched for in the normalized path.
Fixes#3312
It accepts comma separated list of names that are
either string module attributes (ascii_letters,digits, etc)
or are used literally
To enter comma use two commas ',,' somewhere - preferably at the end
Qoutes and double qoutes are not supported
Still compatible with user: but deprecating it so we can have
a matching remote_user: in tasks, cannot be user: because of the
module of the same name. #3932
Signed-off-by: Brian Coca <briancoca+dev@gmail.com>
Reported by Rumen:
TASK: [fail FAIL] *************************************************************
skipping: [hostname.com]
failed: [hostname.com] => {"failed": true}
msg: Failed as requested from task
Traceback (most recent call last):
File "/usr/local/bin/ansible-playbook", line 268, in <module>
sys.exit(main(sys.argv[1:]))
File "/usr/local/bin/ansible-playbook", line 208, in main
pb.run()
File "/Library/Python/2.7/site-packages/ansible/playbook/__init__.py", line 262, in run
if not self._run_play(play):
File "/Library/Python/2.7/site-packages/ansible/playbook/__init__.py", line 580, in _run_play
if (hosts_count - len(host_list)) > int((play.max_fail_pct)/100.0 * hosts_count):
TypeError: object of type 'NoneType' has no len()
Adds original_transport attribute to Runner to track what the original
transport was before it is changed to 'accelerate'.
If using paramiko in original_transport, uses ParamikoConnection. If
not, falls back to SSHConnection like before.
The play was just checking for the presence of the keyword in the
YAML datastructure, and not the value of the field, so doing something
like variable substitution was always causing the play to be accelerated
* Default variables are now fed directly into roles, just like the
other variables, so that roles see their unique values rather
than those set at the global level.
* Role dependency duplicates are now determined by checking the params used
when specifying them as dependencies rather than just on the name of the
role. For example, the following would be included twice without having
to specify "allow_duplicates: true":
dependencies:
- { role: foo, x: 1 }
- { role: foo, x: 2 }
Caveats:
* requiretty must be disabled in the sudoers config
* asking for a password doesn't work yet, so any sudoers users must
be configured with NOPASSWD
* if not starting the daemon as root, the user running the daemon
must have sudoers entries to allow them to run the command as the
target sudo_user
Still needs:
* chunked file transfer/receive
* should probably move all send/recv operations to separate
functions to reduce code duplication
* initial connection setup over ssh? or do we handle that in runner?
This is based somewhat loosely on how Keyczar does things. Their
implementation does things in a much more generic way to allow for more
variance in how the cipher is created, but since we're only using one
key type most of our values are hard-coded. They also add a header to
their messages, which I am not doing (don't see the need for it
currently).
Files were being created in /tmp, but will now be created in $HOME/.ansible/cp/
Addresses CVE-2013-4259: ansible uses a socket with predictable filename in /tmp
The 'always_run' task clause allows one to execute a task even in
check mode.
While here implement Runner.noop_on_check() to check if a runner
really should execute its task, with respect to check mode option
and 'always_run' clause.
Also add the optional 'jinja2' argument to check_conditional() :
it allows to give this function a jinja2 expression without exposing
the 'jinja2_compare' implementation mechanism.
For some reason, ssh seems to ask for password even when
PasswordAuthentication is set to no, adding PreferredAuthentications
with the 2 options removed do the trick.
Tests `test_playbook_undefined_varsX_fail` check if ansible detects
undefined variables when `error_on_undefined_vars` is enabled. These
tests fail without "Improve behavior with error_on_undefined_vars
enabled" patch.
Tests `test_playbook_undefined_varsX_ignore` check if ansible ignores
undefined variables when `error_on_undefined_vars` is disabled.
Also modify PlayBook._run_task_internal() so error_on_undefined_vars is
testable.
Pass fail_on_undefined flag to recursive calls to `template` function,
so more undefined variables are detected.
Works only for Jinja style variables. Undefined legacy variables are
never detected.
Previously, hostvars would only expose a keys() list of hosts that had
been seen yet- however you could explicitly access the host if you knew
the name, and get the content that way. This precludes template code
from being able to safely access information about other hosts if any
limiters/tags were in use.
Additionally, the object was inconsistent for hostvars['myhost'] access
and [x[1] for x in hostvars.items() if x[0] == 'myhost'] access; this is
due to the original derivation from the dict object. .items() would be
handled by dict.items(), using the passed in setup_cache values without
using the actual lookup content.
This patch rebases the class implementation to a py2.6 dictmixin, fixing
those issues and restoring behaviour to match what the docs claim.
For link-local addresses, it is sometimes necessary to append the
interface to use for the ipv6 address. This patch extends the ipv6
regex to allow for '%ifnameX' at the end.
See https://bugzilla.redhat.com/show_bug.cgi?id=136852 for more info
Due to various inconsistencies of ssh and sftp regarding ipv6 and
ipv4 handling, some special arguments must be passed, and the
ipv6 must be passed in a specific format.
testing with a ipv6 :
ansible -u misc -i '[2002::c23e]:22,' '*' -m ping
fail due to parsing of ':' as a separator of port/ip with ipv4.
This commit add support for properly parsing 2002::c23 and the
bracket notation [2002::ce]:2222
The block that added the original list of roles was indented too far,
and was only being reached if a role had dependencies. This resulted
in roles without dependencies from being added to the list of roles.
Credit goes to looped for reporting and diagnosing the issue.
Fixes#3686
Dependencies are enabled by adding a new directory/file named
meta/main.yml to the role. The format of the dependencies are:
dependencies:
- { role: foo, x: 1, y: 2 }
- { role: bar, x: 3, y: 4 }
...
Dependencies inherit variables as they are seen at the time of the
dependency inclusion. For example, if foo(x=1, y=2) has a dependency
on bar(x=3,z=4), then bar will have variables (x=3,y=2,z=4).
Different roles can have dependencies on the same role, and this
variable inheritence allows for the reuse of generic roles quite easily.
For example:
Role 'car' has the following dependencies:
dependencies:
- { role: wheel, n: 1 }
- { role: wheel, n: 2 }
- { role: wheel, n: 3 }
- { role: wheel, n: 4 }
Role 'wheel' has the following dependencies:
dependencies:
- { role: tire }
- { role: brake }
The role 'car' is then used as follows:
- { role: car, type: honda }
And tasks/main.yml in each role simply contains the following:
- name: {{ type }} whatever {{ n }}
command: echo ''
TASK: [honda tire 1]
TASK: [honda brake 1]
TASK: [honda wheel 1]
TASK: [honda tire 2]
TASK: [honda brake 2]
TASK: [honda wheel 2]
TASK: [honda tire 3]
TASK: [honda brake 3]
TASK: [honda wheel 3]
TASK: [honda tire 4]
TASK: [honda brake 4]
TASK: [honda wheel 4]
TASK: [I'm a honda] <- (this is in roles/car/tasks/main.yml)
The idea is that some plugin would not be called in some
specific case, and the callback should decide by itself.
Having a way to globally disable it is much cleaner than
disabling every method one by one on the plugin side.
My use case is for fedora-infrastructure that cannot be run
from git checkout since it try to connect to the message bus,
but another case would be to bootstrap infrastructure, or to
run the code on a test servers without having all the callback
infrastructure setup.
by ensuring all basedirs, plugin paths and extra
paths are handled as absolute paths and are checked
to not add any doubles.
This fixes the corner case where e.g. the user has
an additional plugin path configured to a dir
relative to his playbooks or inventory location,
which also matches the _plugin subdir relative to
one of the basedirs in the play.
For most plugins this doesn't show as an obvious issue
except for callback_plugins, which might fire more
than once. Other plugins (inventory and template
plugins) might unnecessarily be ran twice.
e.g. ansible.cfg has
callback_plugins = ./plays/callback_plugins
and plays/ contains a playbook file:
.
├── ansible.cfg
├── inventory
└── plays
├── callback_plugins
│ └── timestamp.py
└── site.yml
modified: lib/ansible/utils/plugins.py
Previous patch was reverted due to the fact that there was an issue
with the results not always being a dictionary (they're sometimes
a unicode string, ie. when the with_items is used with yum). This
minor change corrects that by checking for a dict object.
to ensure consistent behavior, hosts should look like this:
hosts: webservers:&boston:!rack42
So when applying the host selectors, run those without the "&" first,
then the &s, then the !s.
Closes#3500
If SELinux is enabled and mcstrans is running, daemons are restarted on each
run. After further debugging, it turn out that ansible compare the untranslated
level 's0' with the translated level 'SystemLow' due to mcstrans being running,
which trigger a handler since this is considered as a change.
and the _meta hash contains a "hostvars", don't call --host hostname for any elements
and just serve them directly for performance enhancements with the external inventory
script and a large number of hosts.
Added support of an optional init method for action modules like rsync that need to alter the connection and other inject data before it's established.
Treat errno 13 (permission denied) as one of the special cases in
atomic_move.
This type of error can occur because of sudo'ing to non-root user.
Fixes#3705
Since ansible 1.2, it became possible to place a host_vars
directory in the same directory as a playbook, making it possible
to keep host_vars local to that playbook there. However, due to
python's os.path.dirname, a action such as:
$ ansible-playbook pb.yml
..would not pick up the host_vars as os.path.dirname("pb.yml")
returns "", unlike the unix command dirname that would return
".". Substituting "pb.yml" on the command line with "./pb.yml"
would do the trick, but is not always intuitive. This patch
solves the problem until python solves issue18547 [1].
[1] http://bugs.python.org/issue18547
-c ssh is preferred in most cases if you have ControlPersist available, otherwise if you are comfortable you
can turn off recording while leaving host key checking on, etc.
Use case: e.g. dual homed hosts on production en management network
The inventory_hostname is the regular host name and matches the
dns name on the production network; ansible connects to the host
through a management network; the dns name on the management network
is standardized and equals ${inventory_hostname}-mgt.mynetwork.com
Now this can be configured as the default in group_vars/all:
ansible_ssh_host: {{ inventory_hostname + '-mgt.mynetwork.com' }}
str() throws an UnicodeEncodeError for code points that cannot be
represented in 7-bit ASCII. This makes it impossible to use any
non-ASCII characters in module arguments. Using encode('utf-8')
gives the desired result.
* Moved the --list-hosts option that is common to both `ansible` and
`ansible-playbook` into utils/__init__.py (corrects a FIXME)
* Wrote new help text for the --list-hosts option that makes sense
for both of the commands that it applies to
* Changed the usage argument in `ansible-playbook` so that it is
setup in the base_parser method the same way that it is in
the `ansible` executable
* Updated the help text for several options to correct typos,
clarify meaning, improve readability, or fix grammatical errors.
In the case of `ansible-pull`, I changed the help text so that
it adheres to the same standards as the other executables.
The action doesn't actually change anything on a system, so setting
the status to changed is wrong. add_host is much like set_fact in that
regard.
Since changed is False by default, there is no need to explicity set
it, so just create an empty dict for result and add to it from there.
ansible.constants was calling expanduser (by way of shell_expand_path)
on the entire configured value for the library and *_plugins
configuration values, but these values have always been interpreted as
multiple directories separated by os.pathsep. Thus, if you supplied
multiple directories for one of these values, typically only the first
(at least on *nix) would have e.g. "~" expanded to HOME.
Now PluginLoader does expansion on each individual path in each of
these variables.
A host pattern of the form '!foo' by itself does not work, but
'all:!foo' does. If the first pattern is a negation, this commit
automatically prepends 'all'.
Signed-off-by: martin f. krafft <madduck@madduck.net>
name is used throughout Ansible, it's the "standard". This change
applies that standard to the add_host routine and updates the docs to
reflect that. Related to https://github.com/ansible/ansible/pull/3254
commit c36b66dc952dfff91043ecbca56cf3f1f8f00703
Merge: 240d7bf f4cf934
Author: Michael DeHaan <michael@ansibleworks.com>
Date: Tue Jun 18 13:04:51 2013 -0400
Merge branch 'unevaluated-vars' of git://github.com/lorin/ansible into lorin_undefined
Conflicts:
lib/ansible/runner/__init__.py
commit f4cf93436767f73b62a16067ab5e628830045896
Merge: 2531440 07a1365
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Thu Jun 6 11:07:41 2013 -0400
Merge branch 'devel' into unevaluated-vars
commit 253144045cbafd7d72836f1017c62ac4ba623186
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Thu Jun 6 11:06:37 2013 -0400
Fail template from file on undefined vars
If config option is set, raise an exception if templating from a
file and a variable is undefined.
commit aecb71d8b75257f0f3e11a9b176fc3737aecef8d
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Wed Jun 5 17:12:12 2013 -0400
Add fail_on_undefined flag
Add a fail_on_undefined flag to the template and template_from_string methods.
If this flag is true, then re-raise the ninja2.excpetions.UndefinedError instead of
swallowing it.
commit cbb1808f0585f01536240aee05a1bfd06c4b4647
Merge: d4bbf49 41425fb
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Wed Jun 5 16:14:12 2013 -0400
Merge branch 'devel' into unevaluated-vars
commit d4bbf492b0b63c789d66ab60d0ec634d100fca82
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Mon Jun 3 19:46:13 2013 -0400
template: Raise UndefinedError exception
In template_from_string, raise an undefined error if it occurs.
Have the caller catch it and throw an AnsibleUndefinedVariable
commit c94780280515f1f3756fdc429b2b1e87b365e9b7
Merge: 8d919d6 be33bcf
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Mon Jun 3 10:09:43 2013 -0400
Merge branch 'devel' into unevaluated-vars
commit 8d919d6c97b28a42f47ca7248c542695baf6175f
Merge: 0f68ad8 b8630d2
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Thu May 30 16:27:48 2013 -0400
Merge branch 'devel' into unevaluated-vars
commit 0f68ad8193ac17488e339a258f8c63fdae399c26
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Thu May 30 14:32:03 2013 -0400
Optionally fail task on undefined variables
This patch introduces a new configuration option called
error_on_undefined_vars, which defaults to false.
If this option is set to true, then a task which has unevaluated
variables in its arguments will fail instead of running. Output looks
like this:
TASK: [set rabbitmq password] *************************************************
fatal: [10.20.0.7] => Undefined variables: rabbitmq_user, rabbitmq_password
hardcoded lists in ansible code, just add WITH_ITEMS_USES_LIST in a
comment anywhere, and of course, support recieving params as list.
Signed-off-by: Brian Coca <briancoca+dev@gmail.com>
Previous commit c3659741 expanded sudo_user during task construction,
but this is too early as it does not pick up variables set during
the play.
This commit moves sudo_user expansion to the runner after variables
have been merged.
e.g. db[01:10:3]node-[01:10]
- to do this we split off at the first [...] set, getting the list
of hosts and then repeat until none left.
- also add an optional third parameter which contains the step. (Default: 1)
so range can be [01:10:2] -> 01 03 05 07 09
If someone has a " #" in a quoted var string, it
will interpret that as a comment and refuse to
load the inventory file due to an unbalanced
quote. Noisy failure > unexpected behavior.
Previously setting force=no caused copy to subversively
fail when target did not exist on remote host.
Caused by Runner._remote_md5 returning 1
when files don't exist, rather than 0.
PluginLoader._get_paths, as of 391fb98e, was only finding plug-ins that
were in a subdirectory of one of the basedirs (i.e. in a category
directory). For example, action_plugins/foo.py would never be loaded,
but action_plugins/bar/foo.py would work.
This makes it so that "uncategorized" plug-ins in the top level of a
directory such as action_plugins will be loaded, though plug-ins in a
"category" subdirectory will still be preferred. For example,
action_plugins/bar/foo.py would be preferred over action_plugins/foo.py.
If a variable was provided for an include, in either of these ways:
---
- hosts: all
tasks:
- include: included.yml param=www-data
- include: included.yml
vars:
param: www-data
and then that param was used as the value of sudo_user in the included
tasks:
---
- name: do something as a parameterized sudo_user
command: whoami
sudo: yes
sudo_user: $param
you would receive a "failed to parse: usage: sudo" error back and the
command would not execute.
This seemed to be due to a missing call to template.template somewhere,
because the final value being passed through ssh was still `$param`.
After some digging, the issue seems to instead have been a problem with
providing the wrong context to the template for expansion. Inside the
`Task` logic, it was passing `play.vars` as the context, where
`module_vars` seemed more appropriate. After replacing it, my test case
above ran without issue. There was a comment above suggesting that the
template call might be unnecessary, but removing it made the original
error return, since it is not getting escaped later down the line. I
removed the comment since it was inaccurate.
I tried to actually incorporate my test case above into the test suite
as a regression test, but was unable to figure out how to structure it.
The existing test infrastructure seemed to only be testing for correct
number of counts in things (ok vs. changed, etc.), without regard for
whether the content generated by the command is correct. If there is an
example of a test similar to this one (where I would want to check the
JSON generated to make sure sudo_user had been converted), please let me
know and I will be happy to submit an additional patch.
Excplicity set paramiko's logging level to WARNING.
By default it inherits ansible's DEBUG logging level (set in
callbacks.py) and fills the log file with useless debug messages.
Obviously it only applies if log_path is set in ansible.cfg
Added new parameter 'encrypt' with same semantics from that of
vars_prompt. When encryption is requested a random salt will be
generated and stored along the password in the form:
'<password> salt=<salt>'.
Also store passwords with an ending '\n' for easier looking at files
with console tools. File content was being already rstripped so this
is harmless.
From issue #2820, --start-at-task does not actually run tasks
unless --step is specified. This appears to be because skip_task
is being evaluated as True in PlayBook._run_task(). This patch
ensures skip_task is set to False in the callback.
This is intended to fix#2810. It sets the context of the tmp_dest file
after shutil.move() operation and before os.rename(). This should
retain the selinux context of the file across moves.
There are various cases where a UID to username to UID mapping breaks
down. One UID can be used by two usernames, or no username. If we
always use UIDs internally, then these ambiguous cases won't be a
problem.
The old test used syntax that appeared to be bash-specific and did not
work on platforms where /bin/sh did not point to bash. See issue #2742
where copy to solaris hosts failed with the error:
output: {'stdout': '', 'stderr': '/bin/sh: test: argument expected\n',
'rc': 1}
This fixes#2632. Briefly: specifying things like paths using complex
args in a playbook will make the objects unicode instances. The selinux
module does not accept unicode instances for its char * arguments; it
wants str instances.
Per mpdehaan's comment on #2632 I just went ahead and converted all
paths to UTF-8. I don't know if it would be better to do something like
converting to locale.getpreferredencoding(), but I factored all the
conversions out into new method _to_filesystem_str, so there's only one
place that needs to be changed in the future.
This module allows you to set host facts (or export play variables to the playbook scope if you fancy that).
The module also accepts complex arguments.
```yaml
- action: set_fact fact="something" global_fact="${local_var}"'
- action: set_fact
args:
fact: something
global_fact: ${local_var}
```
- add a skip option so it won't raise an exception if you don't match anything
- make it work as a drop-in replacement for first_available_file
- document in the module comments all of the above cases
AR function was leaving some tmp files behind, want to revert, will have better implementation soon, this is the old way now.
This reverts commit f74a1fa4f0.
Lookup plugins 'sequence' and 'template' now import 'ansible.utils'
appropriately in order to use the 'listify_lookup_plugin_terms'
function.
Also, 'dnstxt' and 'env' now check to see if 'terms' is a string;
without this calls like '{{ lookup('env', 'HOME') }}' fail.
The copy action accepts force=no, which tells it not to replace an
existing file even if it differs from the source. The copy action
plug-in wasn't respecting this option when operated in check mode, so it
would report that changes are necessary in check mode even though copy
would make no changes when run normally.
Runner._remote_md5 was changed to make the logic for setting rc perhaps
a little more clear, and to make sure that rc=0 when the file does not
exist.
Look for a file with the base name of the group/host, first without
a file extension, then with a '.yml' extension, and, finally, with
a '.yaml' extension, loading vars from only the first one found.
As documented in #2623, early variable substitution causes when_
tests to fail and possibly other side effects.
I can see the reason for this early substitution, likely introduced
in 1dfe60a6, to allow many playbook parameters to be templated.
This is a valid goal, but the recursive nature of the utils.template
function means that it goes too far.
At this point removing tasks from the list of parameters to be
substituted seems sufficient to make my tests pass. It may be the
case that other parameters should be excluded, but I suspect not.
Adding a test case. I would prefer to analyse not just the aggregate
statistics but also whether the results are as expected - I can't
see an easy way to do that with the available callbacks at present.
Technically this isn't quite valid YAML when this happens, so we make it valid. This means that if a future commander
API allows save/load it should make sure it does similar processing.
evaluate and replace '$item' with ''. Really it doesn't make sense to include multiple playbooks
via a loop variable, as you can do this with task + with_items already (and it's a simpler code
path). Given this is undocumented, this removes that feature, and we'll consider next how to
also add 'with_items' support directly to roles.
When operating on a unicode string in python 2.6, shlex.split returns
a result that does not work with the file constructor.
To reproduce this requires a task include that is templated (this is
because the templated string is a unicode result, whereas a non-
templated string is a non-unicode string)
[will@centos6.3] $ python
Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import shlex
>>> shlex.split(u'abc')
['a\x00\x00\x00b\x00\x00\x00c\x00\x00\x00']
[will@fedora17] $ python
Python 2.7.3 (default, Jul 24 2012, 10:05:38)
[GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import shlex
>>> shlex.split(u'abc')
['abc']
The proposed fix (coercing the include parameters to string before the
shlex.split) may not be ideal but it does fix the bug for my test case.
python-selinux can't be installed w/ the yum module when missing.
Revert "Added an extra check for target nodes with selinux but without libselinux-python package installed"
This reverts commit 550986b6a1.
Use DEFAULT_EXECUTABLE when no executable is passed to
_low_level_command_exec
Works as a standard constant - can be overridden in all the normal ways
and defaults to /bin/sh
Motiviation is for a user that only has /bin/bash in /etc/sudoers
differnt from the current user. This should enable overrides for user in
.ssh/config w/o breaking any current functionality.
Signed-off-by: Brian Coca <briancoca+dev@gmail.com>
This makes the log message the same, whether it is sent to systemd's
journal or to syslog. It retains the extra fields that are passed to
journal, such as MOUDLE=<name> and additional arguments. Since journal
will reflect messages to syslog, this keeps what goes to syslog
informative instead of the terse 'Ansible module invoked'.
See issue #2461.
The uri module can be configured to abort after a specified timeout if
it cannot connect to the configured uri. This prevents a uri action from
hanging indefinitely when the remote endpoint cannot be reached because
it is unavailable, there is a firewall in place etc. The default behavior
is left unchanged: timeout=None
This change also introduces a new type for module_parameters: int
Code was added to perform conversion from string -> int type in
module_common.py.
The new type was required in order to play nice with httplib2 which
refuses to accept (and convert) anything other than a numeric type for
the timeout value.
