If a user supplies a string in the config (rather than an int), the code
should fix that- or blow up immediately- rather than allowing that value to
work it's way down and break w/in the connection object; when that happens,
the actual error is opaque and requires pdb.set_trace() to run down.
When collecting stdout/stderr *and* feeding data into a Popen instance,
communicate() must be used to avoid a known deadlocking scenario
when data sizes cross PIPE_BUF (which can be as small as 512, although should
be much larger in practice on linux).
This patch also checks specifically for a return code of 255, which
indicates an unknown SSH error of some kind. When that happens, ansible
will now recommend running with -vvvv (if not enabled) or show the
output from 'ssh -vvv' (when it is enabled)
This shouldn't generally be needed unless you're working in an environment
that uses rediculously long FQDNs; if the name is too long, you wind up
hitting unix domain socket filepath limits enforced by ssh.
Two fixes:
* parameter name is selevel, not serange.
* Fix split on selinux context to limit to max of 4 since the selevel
may contain ':' characters. This was fixed in
selinux_default_context() and selinux_context().
Previously, includes had to receive variables via a special 'vars'
field. With this patch, the include syntax becomes a more natural
datastructure without special fields and is more akin to the way
role includes/dependencies work.
Tested with the following playbook:
---
- hosts: localhost
connection: local
tasks:
- { include: inc1.yml, a: 1 }
- include: inc2.yml
b: 2
- include: inc3.yml
with_items:
- x
- y
- z
Fixes#3481
This allows an included path to be relative to a directory, if the
basedir of the original path is a symlink. In that case, the path
is normalized and the file is searched for in the normalized path.
Fixes#3312
It accepts comma separated list of names that are
either string module attributes (ascii_letters,digits, etc)
or are used literally
To enter comma use two commas ',,' somewhere - preferably at the end
Qoutes and double qoutes are not supported
Still compatible with user: but deprecating it so we can have
a matching remote_user: in tasks, cannot be user: because of the
module of the same name. #3932
Signed-off-by: Brian Coca <briancoca+dev@gmail.com>
Reported by Rumen:
TASK: [fail FAIL] *************************************************************
skipping: [hostname.com]
failed: [hostname.com] => {"failed": true}
msg: Failed as requested from task
Traceback (most recent call last):
File "/usr/local/bin/ansible-playbook", line 268, in <module>
sys.exit(main(sys.argv[1:]))
File "/usr/local/bin/ansible-playbook", line 208, in main
pb.run()
File "/Library/Python/2.7/site-packages/ansible/playbook/__init__.py", line 262, in run
if not self._run_play(play):
File "/Library/Python/2.7/site-packages/ansible/playbook/__init__.py", line 580, in _run_play
if (hosts_count - len(host_list)) > int((play.max_fail_pct)/100.0 * hosts_count):
TypeError: object of type 'NoneType' has no len()
Adds original_transport attribute to Runner to track what the original
transport was before it is changed to 'accelerate'.
If using paramiko in original_transport, uses ParamikoConnection. If
not, falls back to SSHConnection like before.
The play was just checking for the presence of the keyword in the
YAML datastructure, and not the value of the field, so doing something
like variable substitution was always causing the play to be accelerated
* Default variables are now fed directly into roles, just like the
other variables, so that roles see their unique values rather
than those set at the global level.
* Role dependency duplicates are now determined by checking the params used
when specifying them as dependencies rather than just on the name of the
role. For example, the following would be included twice without having
to specify "allow_duplicates: true":
dependencies:
- { role: foo, x: 1 }
- { role: foo, x: 2 }
Caveats:
* requiretty must be disabled in the sudoers config
* asking for a password doesn't work yet, so any sudoers users must
be configured with NOPASSWD
* if not starting the daemon as root, the user running the daemon
must have sudoers entries to allow them to run the command as the
target sudo_user
Still needs:
* chunked file transfer/receive
* should probably move all send/recv operations to separate
functions to reduce code duplication
* initial connection setup over ssh? or do we handle that in runner?
This is based somewhat loosely on how Keyczar does things. Their
implementation does things in a much more generic way to allow for more
variance in how the cipher is created, but since we're only using one
key type most of our values are hard-coded. They also add a header to
their messages, which I am not doing (don't see the need for it
currently).
Files were being created in /tmp, but will now be created in $HOME/.ansible/cp/
Addresses CVE-2013-4259: ansible uses a socket with predictable filename in /tmp
The 'always_run' task clause allows one to execute a task even in
check mode.
While here implement Runner.noop_on_check() to check if a runner
really should execute its task, with respect to check mode option
and 'always_run' clause.
Also add the optional 'jinja2' argument to check_conditional() :
it allows to give this function a jinja2 expression without exposing
the 'jinja2_compare' implementation mechanism.
For some reason, ssh seems to ask for password even when
PasswordAuthentication is set to no, adding PreferredAuthentications
with the 2 options removed do the trick.
Tests `test_playbook_undefined_varsX_fail` check if ansible detects
undefined variables when `error_on_undefined_vars` is enabled. These
tests fail without "Improve behavior with error_on_undefined_vars
enabled" patch.
Tests `test_playbook_undefined_varsX_ignore` check if ansible ignores
undefined variables when `error_on_undefined_vars` is disabled.
Also modify PlayBook._run_task_internal() so error_on_undefined_vars is
testable.
Pass fail_on_undefined flag to recursive calls to `template` function,
so more undefined variables are detected.
Works only for Jinja style variables. Undefined legacy variables are
never detected.
Previously, hostvars would only expose a keys() list of hosts that had
been seen yet- however you could explicitly access the host if you knew
the name, and get the content that way. This precludes template code
from being able to safely access information about other hosts if any
limiters/tags were in use.
Additionally, the object was inconsistent for hostvars['myhost'] access
and [x[1] for x in hostvars.items() if x[0] == 'myhost'] access; this is
due to the original derivation from the dict object. .items() would be
handled by dict.items(), using the passed in setup_cache values without
using the actual lookup content.
This patch rebases the class implementation to a py2.6 dictmixin, fixing
those issues and restoring behaviour to match what the docs claim.
For link-local addresses, it is sometimes necessary to append the
interface to use for the ipv6 address. This patch extends the ipv6
regex to allow for '%ifnameX' at the end.
See https://bugzilla.redhat.com/show_bug.cgi?id=136852 for more info
Due to various inconsistencies of ssh and sftp regarding ipv6 and
ipv4 handling, some special arguments must be passed, and the
ipv6 must be passed in a specific format.
testing with a ipv6 :
ansible -u misc -i '[2002::c23e]:22,' '*' -m ping
fail due to parsing of ':' as a separator of port/ip with ipv4.
This commit add support for properly parsing 2002::c23 and the
bracket notation [2002::ce]:2222
The block that added the original list of roles was indented too far,
and was only being reached if a role had dependencies. This resulted
in roles without dependencies from being added to the list of roles.
Credit goes to looped for reporting and diagnosing the issue.
Fixes#3686
Dependencies are enabled by adding a new directory/file named
meta/main.yml to the role. The format of the dependencies are:
dependencies:
- { role: foo, x: 1, y: 2 }
- { role: bar, x: 3, y: 4 }
...
Dependencies inherit variables as they are seen at the time of the
dependency inclusion. For example, if foo(x=1, y=2) has a dependency
on bar(x=3,z=4), then bar will have variables (x=3,y=2,z=4).
Different roles can have dependencies on the same role, and this
variable inheritence allows for the reuse of generic roles quite easily.
For example:
Role 'car' has the following dependencies:
dependencies:
- { role: wheel, n: 1 }
- { role: wheel, n: 2 }
- { role: wheel, n: 3 }
- { role: wheel, n: 4 }
Role 'wheel' has the following dependencies:
dependencies:
- { role: tire }
- { role: brake }
The role 'car' is then used as follows:
- { role: car, type: honda }
And tasks/main.yml in each role simply contains the following:
- name: {{ type }} whatever {{ n }}
command: echo ''
TASK: [honda tire 1]
TASK: [honda brake 1]
TASK: [honda wheel 1]
TASK: [honda tire 2]
TASK: [honda brake 2]
TASK: [honda wheel 2]
TASK: [honda tire 3]
TASK: [honda brake 3]
TASK: [honda wheel 3]
TASK: [honda tire 4]
TASK: [honda brake 4]
TASK: [honda wheel 4]
TASK: [I'm a honda] <- (this is in roles/car/tasks/main.yml)
The idea is that some plugin would not be called in some
specific case, and the callback should decide by itself.
Having a way to globally disable it is much cleaner than
disabling every method one by one on the plugin side.
My use case is for fedora-infrastructure that cannot be run
from git checkout since it try to connect to the message bus,
but another case would be to bootstrap infrastructure, or to
run the code on a test servers without having all the callback
infrastructure setup.
by ensuring all basedirs, plugin paths and extra
paths are handled as absolute paths and are checked
to not add any doubles.
This fixes the corner case where e.g. the user has
an additional plugin path configured to a dir
relative to his playbooks or inventory location,
which also matches the _plugin subdir relative to
one of the basedirs in the play.
For most plugins this doesn't show as an obvious issue
except for callback_plugins, which might fire more
than once. Other plugins (inventory and template
plugins) might unnecessarily be ran twice.
e.g. ansible.cfg has
callback_plugins = ./plays/callback_plugins
and plays/ contains a playbook file:
.
├── ansible.cfg
├── inventory
└── plays
├── callback_plugins
│ └── timestamp.py
└── site.yml
modified: lib/ansible/utils/plugins.py
Previous patch was reverted due to the fact that there was an issue
with the results not always being a dictionary (they're sometimes
a unicode string, ie. when the with_items is used with yum). This
minor change corrects that by checking for a dict object.
to ensure consistent behavior, hosts should look like this:
hosts: webservers:&boston:!rack42
So when applying the host selectors, run those without the "&" first,
then the &s, then the !s.
Closes#3500
If SELinux is enabled and mcstrans is running, daemons are restarted on each
run. After further debugging, it turn out that ansible compare the untranslated
level 's0' with the translated level 'SystemLow' due to mcstrans being running,
which trigger a handler since this is considered as a change.
and the _meta hash contains a "hostvars", don't call --host hostname for any elements
and just serve them directly for performance enhancements with the external inventory
script and a large number of hosts.
Added support of an optional init method for action modules like rsync that need to alter the connection and other inject data before it's established.
Treat errno 13 (permission denied) as one of the special cases in
atomic_move.
This type of error can occur because of sudo'ing to non-root user.
Fixes#3705
Since ansible 1.2, it became possible to place a host_vars
directory in the same directory as a playbook, making it possible
to keep host_vars local to that playbook there. However, due to
python's os.path.dirname, a action such as:
$ ansible-playbook pb.yml
..would not pick up the host_vars as os.path.dirname("pb.yml")
returns "", unlike the unix command dirname that would return
".". Substituting "pb.yml" on the command line with "./pb.yml"
would do the trick, but is not always intuitive. This patch
solves the problem until python solves issue18547 [1].
[1] http://bugs.python.org/issue18547
-c ssh is preferred in most cases if you have ControlPersist available, otherwise if you are comfortable you
can turn off recording while leaving host key checking on, etc.
Use case: e.g. dual homed hosts on production en management network
The inventory_hostname is the regular host name and matches the
dns name on the production network; ansible connects to the host
through a management network; the dns name on the management network
is standardized and equals ${inventory_hostname}-mgt.mynetwork.com
Now this can be configured as the default in group_vars/all:
ansible_ssh_host: {{ inventory_hostname + '-mgt.mynetwork.com' }}
str() throws an UnicodeEncodeError for code points that cannot be
represented in 7-bit ASCII. This makes it impossible to use any
non-ASCII characters in module arguments. Using encode('utf-8')
gives the desired result.
* Moved the --list-hosts option that is common to both `ansible` and
`ansible-playbook` into utils/__init__.py (corrects a FIXME)
* Wrote new help text for the --list-hosts option that makes sense
for both of the commands that it applies to
* Changed the usage argument in `ansible-playbook` so that it is
setup in the base_parser method the same way that it is in
the `ansible` executable
* Updated the help text for several options to correct typos,
clarify meaning, improve readability, or fix grammatical errors.
In the case of `ansible-pull`, I changed the help text so that
it adheres to the same standards as the other executables.
The action doesn't actually change anything on a system, so setting
the status to changed is wrong. add_host is much like set_fact in that
regard.
Since changed is False by default, there is no need to explicity set
it, so just create an empty dict for result and add to it from there.
ansible.constants was calling expanduser (by way of shell_expand_path)
on the entire configured value for the library and *_plugins
configuration values, but these values have always been interpreted as
multiple directories separated by os.pathsep. Thus, if you supplied
multiple directories for one of these values, typically only the first
(at least on *nix) would have e.g. "~" expanded to HOME.
Now PluginLoader does expansion on each individual path in each of
these variables.
A host pattern of the form '!foo' by itself does not work, but
'all:!foo' does. If the first pattern is a negation, this commit
automatically prepends 'all'.
Signed-off-by: martin f. krafft <madduck@madduck.net>
name is used throughout Ansible, it's the "standard". This change
applies that standard to the add_host routine and updates the docs to
reflect that. Related to https://github.com/ansible/ansible/pull/3254
commit c36b66dc952dfff91043ecbca56cf3f1f8f00703
Merge: 240d7bf f4cf934
Author: Michael DeHaan <michael@ansibleworks.com>
Date: Tue Jun 18 13:04:51 2013 -0400
Merge branch 'unevaluated-vars' of git://github.com/lorin/ansible into lorin_undefined
Conflicts:
lib/ansible/runner/__init__.py
commit f4cf93436767f73b62a16067ab5e628830045896
Merge: 2531440 07a1365
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Thu Jun 6 11:07:41 2013 -0400
Merge branch 'devel' into unevaluated-vars
commit 253144045cbafd7d72836f1017c62ac4ba623186
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Thu Jun 6 11:06:37 2013 -0400
Fail template from file on undefined vars
If config option is set, raise an exception if templating from a
file and a variable is undefined.
commit aecb71d8b75257f0f3e11a9b176fc3737aecef8d
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Wed Jun 5 17:12:12 2013 -0400
Add fail_on_undefined flag
Add a fail_on_undefined flag to the template and template_from_string methods.
If this flag is true, then re-raise the ninja2.excpetions.UndefinedError instead of
swallowing it.
commit cbb1808f0585f01536240aee05a1bfd06c4b4647
Merge: d4bbf49 41425fb
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Wed Jun 5 16:14:12 2013 -0400
Merge branch 'devel' into unevaluated-vars
commit d4bbf492b0b63c789d66ab60d0ec634d100fca82
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Mon Jun 3 19:46:13 2013 -0400
template: Raise UndefinedError exception
In template_from_string, raise an undefined error if it occurs.
Have the caller catch it and throw an AnsibleUndefinedVariable
commit c94780280515f1f3756fdc429b2b1e87b365e9b7
Merge: 8d919d6 be33bcf
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Mon Jun 3 10:09:43 2013 -0400
Merge branch 'devel' into unevaluated-vars
commit 8d919d6c97b28a42f47ca7248c542695baf6175f
Merge: 0f68ad8 b8630d2
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Thu May 30 16:27:48 2013 -0400
Merge branch 'devel' into unevaluated-vars
commit 0f68ad8193ac17488e339a258f8c63fdae399c26
Author: Lorin Hochstein <lorin@nimbisservices.com>
Date: Thu May 30 14:32:03 2013 -0400
Optionally fail task on undefined variables
This patch introduces a new configuration option called
error_on_undefined_vars, which defaults to false.
If this option is set to true, then a task which has unevaluated
variables in its arguments will fail instead of running. Output looks
like this:
TASK: [set rabbitmq password] *************************************************
fatal: [10.20.0.7] => Undefined variables: rabbitmq_user, rabbitmq_password
hardcoded lists in ansible code, just add WITH_ITEMS_USES_LIST in a
comment anywhere, and of course, support recieving params as list.
Signed-off-by: Brian Coca <briancoca+dev@gmail.com>
Previous commit c3659741 expanded sudo_user during task construction,
but this is too early as it does not pick up variables set during
the play.
This commit moves sudo_user expansion to the runner after variables
have been merged.
e.g. db[01:10:3]node-[01:10]
- to do this we split off at the first [...] set, getting the list
of hosts and then repeat until none left.
- also add an optional third parameter which contains the step. (Default: 1)
so range can be [01:10:2] -> 01 03 05 07 09
If someone has a " #" in a quoted var string, it
will interpret that as a comment and refuse to
load the inventory file due to an unbalanced
quote. Noisy failure > unexpected behavior.
Previously setting force=no caused copy to subversively
fail when target did not exist on remote host.
Caused by Runner._remote_md5 returning 1
when files don't exist, rather than 0.
PluginLoader._get_paths, as of 391fb98e, was only finding plug-ins that
were in a subdirectory of one of the basedirs (i.e. in a category
directory). For example, action_plugins/foo.py would never be loaded,
but action_plugins/bar/foo.py would work.
This makes it so that "uncategorized" plug-ins in the top level of a
directory such as action_plugins will be loaded, though plug-ins in a
"category" subdirectory will still be preferred. For example,
action_plugins/bar/foo.py would be preferred over action_plugins/foo.py.
If a variable was provided for an include, in either of these ways:
---
- hosts: all
tasks:
- include: included.yml param=www-data
- include: included.yml
vars:
param: www-data
and then that param was used as the value of sudo_user in the included
tasks:
---
- name: do something as a parameterized sudo_user
command: whoami
sudo: yes
sudo_user: $param
you would receive a "failed to parse: usage: sudo" error back and the
command would not execute.
This seemed to be due to a missing call to template.template somewhere,
because the final value being passed through ssh was still `$param`.
After some digging, the issue seems to instead have been a problem with
providing the wrong context to the template for expansion. Inside the
`Task` logic, it was passing `play.vars` as the context, where
`module_vars` seemed more appropriate. After replacing it, my test case
above ran without issue. There was a comment above suggesting that the
template call might be unnecessary, but removing it made the original
error return, since it is not getting escaped later down the line. I
removed the comment since it was inaccurate.
I tried to actually incorporate my test case above into the test suite
as a regression test, but was unable to figure out how to structure it.
The existing test infrastructure seemed to only be testing for correct
number of counts in things (ok vs. changed, etc.), without regard for
whether the content generated by the command is correct. If there is an
example of a test similar to this one (where I would want to check the
JSON generated to make sure sudo_user had been converted), please let me
know and I will be happy to submit an additional patch.
Excplicity set paramiko's logging level to WARNING.
By default it inherits ansible's DEBUG logging level (set in
callbacks.py) and fills the log file with useless debug messages.
Obviously it only applies if log_path is set in ansible.cfg
Added new parameter 'encrypt' with same semantics from that of
vars_prompt. When encryption is requested a random salt will be
generated and stored along the password in the form:
'<password> salt=<salt>'.
Also store passwords with an ending '\n' for easier looking at files
with console tools. File content was being already rstripped so this
is harmless.
From issue #2820, --start-at-task does not actually run tasks
unless --step is specified. This appears to be because skip_task
is being evaluated as True in PlayBook._run_task(). This patch
ensures skip_task is set to False in the callback.
This is intended to fix#2810. It sets the context of the tmp_dest file
after shutil.move() operation and before os.rename(). This should
retain the selinux context of the file across moves.
There are various cases where a UID to username to UID mapping breaks
down. One UID can be used by two usernames, or no username. If we
always use UIDs internally, then these ambiguous cases won't be a
problem.
The old test used syntax that appeared to be bash-specific and did not
work on platforms where /bin/sh did not point to bash. See issue #2742
where copy to solaris hosts failed with the error:
output: {'stdout': '', 'stderr': '/bin/sh: test: argument expected\n',
'rc': 1}
This fixes#2632. Briefly: specifying things like paths using complex
args in a playbook will make the objects unicode instances. The selinux
module does not accept unicode instances for its char * arguments; it
wants str instances.
Per mpdehaan's comment on #2632 I just went ahead and converted all
paths to UTF-8. I don't know if it would be better to do something like
converting to locale.getpreferredencoding(), but I factored all the
conversions out into new method _to_filesystem_str, so there's only one
place that needs to be changed in the future.
This module allows you to set host facts (or export play variables to the playbook scope if you fancy that).
The module also accepts complex arguments.
```yaml
- action: set_fact fact="something" global_fact="${local_var}"'
- action: set_fact
args:
fact: something
global_fact: ${local_var}
```
- add a skip option so it won't raise an exception if you don't match anything
- make it work as a drop-in replacement for first_available_file
- document in the module comments all of the above cases
AR function was leaving some tmp files behind, want to revert, will have better implementation soon, this is the old way now.
This reverts commit f74a1fa4f0.
Lookup plugins 'sequence' and 'template' now import 'ansible.utils'
appropriately in order to use the 'listify_lookup_plugin_terms'
function.
Also, 'dnstxt' and 'env' now check to see if 'terms' is a string;
without this calls like '{{ lookup('env', 'HOME') }}' fail.
The copy action accepts force=no, which tells it not to replace an
existing file even if it differs from the source. The copy action
plug-in wasn't respecting this option when operated in check mode, so it
would report that changes are necessary in check mode even though copy
would make no changes when run normally.
Runner._remote_md5 was changed to make the logic for setting rc perhaps
a little more clear, and to make sure that rc=0 when the file does not
exist.
Look for a file with the base name of the group/host, first without
a file extension, then with a '.yml' extension, and, finally, with
a '.yaml' extension, loading vars from only the first one found.
As documented in #2623, early variable substitution causes when_
tests to fail and possibly other side effects.
I can see the reason for this early substitution, likely introduced
in 1dfe60a6, to allow many playbook parameters to be templated.
This is a valid goal, but the recursive nature of the utils.template
function means that it goes too far.
At this point removing tasks from the list of parameters to be
substituted seems sufficient to make my tests pass. It may be the
case that other parameters should be excluded, but I suspect not.
Adding a test case. I would prefer to analyse not just the aggregate
statistics but also whether the results are as expected - I can't
see an easy way to do that with the available callbacks at present.
Technically this isn't quite valid YAML when this happens, so we make it valid. This means that if a future commander
API allows save/load it should make sure it does similar processing.
