Allow security tokens and profiles to be used as arguments
to the 'common' ec2 modules
Mostly refactoring to provide two new methods,
`get_aws_connection_info`, which results in a dict that can be
passed through to the boto `connect_to_region` calls, and
`connect_to_aws` that can pass that dict through to the
`connect_to_region` method of the appropriate module.
Tidied up some variable names
Works around boto/boto#2100
profiles don't work with boto < 2.24, but this detects for that
and fails with an appropriate message. It is designed to work
if profile is not passed but boto < 2.24 is installed.
Modifications to allow empty aws auth variables to be passed
(this is useful if wanting to have the keys as an optional
parameter in ec2 calls - if set, use this value, if not set,
use boto config or env variables)
Reworked validate_certs improvements to work with refactoring
Added documentation for profile and security_token to affected modules
In order to simplify the workflow with the GCE modules, it's now
possible to add the parameters and project name as arguments to the
various GCE modules.
The inventory plugin also returns the IP of the host in
`ansible_ssh_host` so that you don't have to specify IPs into the
inventory file.
Some update to the documentation are also added.
Closes#5583.
It came up that fixing this unit test may relate to another ticket that is open. This work allows us to uncomment this unit test by fixing how we pars variables allowing a quoted variable to contain a '#'.
Work also went into cleaning up some of the test data to clarify what was working.
Lastly work went into cleaning up formatting so that the code is easily read.
Add support for checking host against global known host files.
The effect of this is that before this fix if files are spread across the known_hosts file but not in the ~/known_hosts file the hosts will execute sequentially. This PR augments the functionality so that all of the knowns hosts will execute in parallel.
##### Issue Type:
Bugfix Pull Request
##### Ansible Version:
ansible 1.4.3
##### Environment:
N/A
##### Summary:
We are using a wrapper python script to run ansible-playbook. We use subprocess to execute and print the stdout as and when its written. Problem is when we use pause it doesn't display the prompt string as raw_input does not flush stdout before reading from stdin.
It looks like a dirty fix to add "\n" to the prompt string but i don't see any other way to over come this. If anyone else have a better fix please do propose/suggest.
##### Steps To Reproduce:
```yaml
#File: test_play.yml
- name: Test
hosts: $nodes
gather_facts: false
tasks:
- name: Waiting for User
local_action: pause prompt="Do you want to continue (yes/no)? "
```
```python
#!/usr/bin/env python
#File: test.py
import shlex, subprocess
def run_process(process):
process = process.encode("utf-8")
command = shlex.split(process)
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in iter(p.stdout.readline, b''):
print line,
cmd = "/usr/bin/python -u /usr/bin/ansible-playbook -i hosts.txt test_play.yml -e 'nodes=local'"
run_process(cmd)
```
```
shell $ python test.py
```
##### Expected Results:
```
PLAY [Test] *******************************************************************
TASK: [Waiting for User] ******************************************************
[localhost]
Do you want to continue (yes/no)? :
```
##### Actual Results:
```
PLAY [Test] *******************************************************************
TASK: [Waiting for User] ******************************************************
[localhost]
```
Addresses GH-5116.
It comes up that when parsing json that if you are missing the last double quote on the last variable and the next line is just '}' we will get an out of range error. In this instance we will also then make sure that the line is long enough to have two colons.
Create a lookup plugin named dict that can be used to loop over hashes.
It converts a dict into a list of key-value pairs, with attributes named
"key" and "value." Also adds a brief explanation and simple example to
the docs.
Signed-off-by: Kent R. Spillner <kspillner@acm.org>
Bugfixes:
* the remote_src param was not being converted to a boolean correctly,
resulting in it never being used by the module as the default behavior
was remote_src=True (issue #5581)
* the remote_src param was not listed in the generic file params, leading
to a failure when the above bug regarding remote_src was fixed
* the delimiter should always end with a newline to ensure that the file
fragments do not run together on one line
Fixes#5581
This occurred when a hash would be passed in via extra args and the
hash variable behavior was set to 'merge', which resulted in the
variable from extra args replacing the playbook variable.
When content is processed and found to be valid JSON it is decoded into a dict. To write it out to a file we need to encode it back into a string.
Addresses GH-5914.
When disabled, the boto connection will be instantiated without validating
the SSL certificate from the target endpoint. This allows the modules to connect
to Eucalyptus instances running with self-signed certs without errors.
Fixes#3978
There is a bit going on with the changes here. Most of the changes are cleanup of files so that they line up with the standard files.
PR #5136 was merged into the current devel and brought up to working order. A few bug fixes had to be done to get the code to test correctly. Thanks out to @pib!
Issue #5431 was not able to be confirmed as it behaved as expected with a sudo user.
Tests were added via a playbook with archive files to verify functionality.
All tests fire clean including custom playbooks across multiple linux and solaris systems.
It turns out that some of the assumptions in #5885 were slightly off. The previous fix relied on a call to the module to creat a tmp_path. This is insufficent as there are few cases that we need to have the tmp directory before we make the module call. If we don't have a tmp_path before we do a recursive call or when we find a file that does not match the remote md5 hash we need to create a tmp directory. Also we are not more percise when we will need to clean up the remote tmp_path.
This doesn't account for boto configs where e.g. RDS has one
default region and EC2 another - all will default to `ec2_region_name`.
However, this is just handy to allow an easy site wide default
region if existing configuration already relies on it.
Modules can be improved to mention this in the documentation and
turn off required=True where needed. But it works with `ec2`
and `ec2_vol` without change.
Refactor the currently well-factored ec2 modules (i.e. those that already use ec2_connect) to
have a common argument spec. The idea is that new modules can use this spec without duplication
of code, and that new functionality can be added to the ec2 connection code (e.g. security
token argument)
We break the read while loop after waiting "the end of the process" and
the pipes are empty, otherwise we do another select that waits all the
timeout.
The copy action_plugin is not easy to read. Part of this commit is taking that file, restructuring it, and adding comments. No functionality changed in how it interacts with the world.
The fix for #5739 ends up being the assumption that there is a cleanup 'rm -rf' that happens at the end of the copy loop. This was not the fact before and we made a bunch of tmp directories that we hoped would end up being cleaned up. Now we just use the tmp directory that the runner provides and cleanup inline if it is a single file to be coppied or after the loop if it is a recursive copy.
As a part of this we did end up having to change runner to provide a flag so that we could short the inline tmp directory removal. This flag defaults to True so it will not change the behavior of other modules that are being called.
In particular, do not rely on the $USER environment variable always existing.
tmux for example seems to clear it, causing lots of invalid messages:
"previous known host file not found"
This broke in commit 80fd22dc, but instead of reverting that commit, we now
fall back to expanding just ~ when $USER is not set.
su_user_var. My last PR was only half merged, and when the bug fix for
the su/su_pass typo was merged, the removed line in this commit was
mistakenly reintroduced.
this variable has the 'current host list' to be executed over in the
play. Useful when using --limit to not iterate over hosts not included
in play in templates or with_items.
Signed-off-by: Brian Coca <briancoca+dev@gmail.com>
The ansible remote port should be None, not 22. Having a default value
of 22 means that '-o Port 22' will be appended to the ssh connection
all of the time. This is incorrect as when one would like to use
something like an ssh configuration file (-F) that sets the port to
something other than 22.
Part of this change requires that we check that, in get_config, the
value is not None before trying to cast it into an integer or float.
As part of 94f3b9bfab the code was changed to support dynamically adding localhost to the inventory. This change introduced an crash when run via ansible-pull
```
Starting ansible-pull at 2014-01-20 23:09:57
Traceback (most recent call last):
File "/tmp/ansible/bin/ansible", line 157, in <module>
(runner, results) = cli.run(options, args)
File "/tmp/ansible/bin/ansible", line 82, in run
hosts = inventory_manager.list_hosts(pattern)
File "/tmp/ansible/lib/ansible/inventory/__init__.py", line 372, in list_hosts
result = [ h.name for h in self.get_hosts(pattern) ]
File "/tmp/ansible/lib/ansible/inventory/__init__.py", line 136, in get_hosts
subset = self._get_hosts(self._subset)
File "/tmp/ansible/lib/ansible/inventory/__init__.py", line 177, in _get_hosts
that = self.__get_hosts(p)
File "/tmp/ansible/lib/ansible/inventory/__init__.py", line 198, in __get_hosts
hpat = self._hosts_in_unenumerated_pattern(name)
File "/tmp/ansible/lib/ansible/inventory/__init__.py", line 275, in _hosts_in_unenumerated_pattern
ungrouped.add_host(new_host)
AttributeError: 'NoneType' object has no attribute 'add_host'
```
The root cause is there is no group for the host to be added to. I fixed this case by creating the ungrouped group when it doesn't exist and then adding the host to the newly added group. This fixes the regression for me.
Operate on that play attribute to make things faster for larger
inventories. Instead of making a round trip through inventory.list_hosts
and working through some lengthy list comprehensions over and over
again, calculate the potenital hosts for a play once, then reduce from
it the unavailable hosts when necessary.
Also moves how the %fail is done. The host count is a play level count
of available hosts, which then is compared after each task to the
current number of available hosts for the play. This used to get a new
count every task which was also time expensive.
1. if accept_hostkey is false, no matter if the host key is known or not, it will fail.
2. We don't check for the host key in /etc/ssh/ssh_known_hosts
This fixes both of those issues.
The _list_available_hosts call can be lengthy, and in the case where
gather_facts is disabled the call is pointless. So re-arrange the logic
to return early from _do_setup_step when gather_facts is false.
Users of these features should use "when:" as documented at docs.ansible.com.
Similarly, include + with_items has been removed. The solution is to loop
inside the task files, see with_nested / with_together, etc.
