On Python 2, shlex.split() raises if you pass it a unicode object with
non-ASCII characters in it. The Ansible codebase copes by explicitly
converting the string using to_bytes() before passing it to
shlex.split().
On Python 3, shlex.split() raises ('bytes' object has no attribute 'read')
if you pass a bytes object. Oops.
This commit introduces a new wrapper function, shlex_split, that
transparently performs the to_bytes/to_unicode conversions only on
Python 2.
Currently I've only converted one call site (the one that was causing a
unit test to fail on Python 3). If this approach is deemed suitable,
I'll convert them all.
There were no inventory-specific unit tests earlier, so we add a new
directory for them with some initial low-level tests of _split_pattern
with various valid and deprecated pattern strings.
Labels must start with an alphanumeric character, may contain
alphanumeric characters or hyphens, but must not end with a hyphen.
We enforce those rules, but allow underscores wherever hyphens are
accepted, and allow alphanumeric ranges anywhere.
We relax the definition of "alphanumeric" to include Unicode characters
even though such inventory hostnames cannot be used in practice unless
an ansible_ssh_host is set for each of them.
We still don't enforce length restrictions—the fact that we have to
accept ranges makes it more complex, and it doesn't seem especially
worthwhile.
This adds a parse_address(pattern) utility function that returns
(host,port), and uses it wherever where we accept IPv4 and IPv6
addresses and hostnames (or host patterns): the inventory parser
the the add_host action plugin.
It also introduces a more extensive set of unit tests that supersedes
the old add_host unit tests (which didn't actually test add_host, but
only the parsing function).
There was code to support set literals (on Python 2.7 and newer), but it
was buggy: SAFE_NODES.union() doesn't modify SAFE_NODES in place,
instead it returns a new set object that is then silently discarded.
I added a unit test and fixed the code. I also changed the version
check to use sys.version_tuple instead of a string comparison, for
consistency with the subsequent Python 3.4 version check that I added in
the previous commit.
The full error was
======================================================================
ERROR: test_task_executor_execute (units.executor.test_task_executor.TestTaskExecutor)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/mg/src/ansible/test/units/executor/test_task_executor.py", line 252, in test_task_executor_execute
mock_action.run.return_value = dict(ansible_facts=dict())
File "/home/mg/src/ansible/lib/ansible/executor/task_executor.py", line 317, in _execute
if self._task.async > 0:
TypeError: unorderable types: MagicMock() > int()
----------------------------------------------------------------------
Experiments show that Python 2 MagicMock() > 0 is true, so I'm setting
the async property on mock_task to 1. (If I set it to 0, the test fails
anyway.)
Required some rewiring in inventory code to make sure we're using
the DataLoader class for some data file operations, which makes mocking
them much easier.
Also identified two corner cases not currently handled by the code, related
to inventory variable sources and which one "wins". Also noticed we weren't
properly merging variables from multiple group/host_var file locations
(inventory directory vs. playbook directory locations) so fixed as well.
Replace .iteritems() with six.iteritems() everywhere except in
module_utils (because there's no 'six' on the remote host). And except
in lib/ansible/galaxy/data/metadata_template.j2, because I'm not sure
six is available there.
ansible is passing unicode arond internally so we should test the same
data.
* Add a zero length test for _count_newlines and fix the zero newlines
test to have no newlines.
Note that this test was broken in devel because it was really just
duplicating the AES256 test because setting v.cipher_name to 'AES'
no longer selected AES after it was de-write-whitelisted.
Now that we've removed the VaultAES encryption code, we embed static
output from an earlier version and test that we can decrypt it.
