ansible/community.general
1
0
Fork 0
mirror of https://github.com/ansible-collections/community.general.git synced 2024-09-14 20:13:21 +02:00
Code Releases Activity

Add support for Windows hosts in the SSH connection plugin (#47732)

Browse source 
* Add support for Windows hosts in the SSH connection plugin

* fix Python 2.6 unit test and sanity issues

* fix up connection tests in CI, disable SCP for now

* ensure we don't pollute the existing environment during the test

* Add connection_windows_ssh to classifier

* use test dir for inventory file

* Required powershell as default shell and fix tests

* Remove exlicit become_methods on connection

* clarify console encoding comment

* ignore recent SCP errors in integration tests

* Add cmd shell type and added more tests

* Fix some doc issues

* revises windows faq

* add anchors for windows links

* revises windows setup page

* Update changelogs/fragments/windows-ssh.yaml

Co-Authored-By: jborean93 <jborean93@gmail.com>
This commit is contained in:
Jordan Borean 2019-03-08 10:38:02 +10:00 committed by Matt Davis
parent cdf475e830
commit 8ef2e6da05
24 changed files with 657 additions and 143 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
changelogs/fragments/windows-ssh.yaml Normal file
View file

@ -0,0 +1,2 @@
minor_changes:
- Added experimental support for connecting to Windows hosts over SSH using ``ansible_shell_type=cmd`` or ``ansible_shell_type=powershell``

2
docs/docsite/rst/user_guide/windows.rst
View file

@ -1,3 +1,5 @@
.. _windows:
Windows Guides
``````````````

105
docs/docsite/rst/user_guide/windows_faq.rst
View file

@ -1,3 +1,5 @@
.. _windows_faq:
Windows Frequently Asked Questions
==================================
@ -6,7 +8,7 @@ their answers.
.. note:: This document covers questions about managing Microsoft Windows servers with Ansible.
For questions about Ansible Core, please see the
:ref:`FAQ page <ansible_faq>`.
:ref:`general FAQ page <ansible_faq>`.
Does Ansible work with Windows XP or Server 2003?
``````````````````````````````````````````````````
@ -23,18 +25,18 @@ supported operating system versions are:
* Windows 10
Ansible also has minimum PowerShell version requirements - please see
:doc:`windows_setup` for the latest information.
:ref:`windows_setup` for the latest information.
Can I Manage Windows Nano Server?
`````````````````````````````````
Can I manage Windows Nano Server with Ansible?
``````````````````````````````````````````````
Windows Nano Server is not currently supported by Ansible, since it does
not have access to the full .NET Framework that is used by the majority of the
modules and internal components.
Can Ansible run on Windows?
```````````````````````````
No, Ansible cannot run on a Windows host natively and can only manage Windows hosts,
but Ansible can be run under the Windows Subsystem for Linux (WSL).
No, Ansible can only manage Windows hosts. Ansible cannot run on a Windows host
natively, though it can run under the Windows Subsystem for Linux (WSL).
.. note:: The Windows Subsystem for Linux is not supported by Ansible and
should not be used for production systems.
@ -60,16 +62,20 @@ installed version and then clone the git repo.
# To enable Ansible on login, run the following
echo ". ~/ansible/hacking/env-setup -q' >> ~/.bashrc
Can I use SSH keys to authenticate?
```````````````````````````````````
Windows uses WinRM as the transport protocol. WinRM supports a wide range of
authentication options. The closet option to SSH keys is to use the certificate
authentication option which maps an X509 certificate to a local user.
Can I use SSH keys to authenticate to Windows hosts?
````````````````````````````````````````````````````
SSH keys are not supported when using the WinRM or PSRP connection plugins.
These connection plugins support X509 certificates for authentication instead
of the SSH key pairs that SSH supports.
The way that these certificates are generated and mapped to a user is different
from the SSH implementation; consult the :doc:`windows_winrm` documentation for
The way X509 certificates are generated and mapped to a user is different
from the SSH implementation; consult the :ref:`windows_winrm` documentation for
more information.
Ansible 2.8 has added experimental support for using the SSH connection plugin,
which supports authentication with SSH keys, to connect to Windows servers. See `this question <windows_faq_ssh>`
for more information.
.. _windows_faq_winrm:
Why can I run a command locally that does not work under Ansible?
@ -93,7 +99,7 @@ Some ways to bypass these restrictions are to:
* Use ``become``, which runs a command as it would when run locally. This will
bypass most WinRM restrictions, as Windows is unaware the process is running
under WinRM when ``become`` is used. See the :doc:`become` documentation for more
under WinRM when ``become`` is used. See the :ref:`become` documentation for more
information.
* Use a scheduled task, which can be created with ``win_scheduled_task``. Like
@ -107,15 +113,15 @@ Some ways to bypass these restrictions are to:
authentication option that supports credential delegation can be used. Both
CredSSP and Kerberos with credential delegation enabled can support this.
See :doc:`become` more info on how to use become. The limitations section at
:doc:`windows_winrm` has more details around WinRM limitations.
See :ref:`become` more info on how to use become. The limitations section at
:ref:`windows_winrm` has more details around WinRM limitations.
This program won't install with Ansible
```````````````````````````````````````
This program won't install on Windows with Ansible
``````````````````````````````````````````````````
See :ref:`this question <windows_faq_winrm>` for more information about WinRM limitations.
What modules are available?
```````````````````````````
What Windows modules are available?
```````````````````````````````````
Most of the Ansible modules in Ansible Core are written for a combination of
Linux/Unix machines and arbitrary web services. These modules are written in
Python and most of them do not work on Windows.
@ -147,39 +153,54 @@ In addition, the following Ansible Core modules/action-plugins work with Windows
* template (also: win_template)
* wait_for_connection
Can I run Python modules?
`````````````````````````
Can I run Python modules on Windows hosts?
``````````````````````````````````````````
No, the WinRM connection protocol is set to use PowerShell modules, so Python
modules will not work. A way to bypass this issue to use
``delegate_to: localhost`` to run a Python module on the Ansible controller.
This is useful if during a playbook, an external service needs to be contacted
and there is no equivalent Windows module available.
Can I connect over SSH?
```````````````````````
Microsoft has announced and is developing a fork of OpenSSH for Windows that
allows remote manage of Windows servers through the SSH protocol instead of
WinRM. While this can be installed and used right now for normal SSH clients,
it is still in beta from Microsoft and the required functionality has not been
developed within Ansible yet.
.. _winrm_faq_ssh:
There are future plans on adding this feature and this page will be updated
once more information can be shared.
Can I connect to Windows hosts over SSH?
````````````````````````````````````````
Ansible 2.8 has added experimental support for using the SSH connection plugin
to manage Windows hosts. To connect to Windows hosts over SSH, you must install and configure the `Win32-OpenSSH <https://github.com/PowerShell/Win32-OpenSSH>`_
fork that is in development with Microsoft on
the Windows host(s). While most of the basics should work with SSH,
``Win32-OpenSSH`` is rapidly changing, with new features added and bugs
fixed in every release. It is highly recommend you install the latest release
of ``Win32-OpenSSH`` from the GitHub Releases page when using it with Ansible
on Windows hosts.
Why is connecting to the host via ssh failing?
``````````````````````````````````````````````
When trying to connect to a Windows host and the output error indicates that
SSH was used, then this is an indication that the connection vars are not set
properly or the host is not inheriting them correctly.
To use SSH as the connection to a Windows host, set the following variables in
the inventory::
ansible_connection=ssh
# Set either cmd or powershell not both
ansible_shell_type=cmd
# ansible_shell_type=powershell
The value for ``ansible_shell_type`` should either be ``cmd`` or ``powershell``.
Use ``cmd`` if the ``DefaultShell`` has not been configured on the SSH service
and ``powershell`` if that has been set as the ``DefaultShell``.
Why is connecting to a Windows host via SSH failing?
````````````````````````````````````````````````````
Unless you are using ``Win32-OpenSSH`` as described above, you must connect to
Windows hosts using :ref:`windows_winrm`. If your Ansible output indicates that
SSH was used, either you did not set the connection vars properly or the host is not inheriting them correctly.
Make sure ``ansible_connection: winrm`` is set in the inventory for the Windows
host.
host(s).
Why are my credentials being rejected?
``````````````````````````````````````
This can be due to a myriad of reasons unrelated to incorrect credentials.
See HTTP 401/Credentials Rejected at :doc:`windows_setup` for a more detailed
See HTTP 401/Credentials Rejected at :ref:`windows_setup` for a more detailed
guide of this could mean.
Why am I getting an error SSL CERTIFICATE_VERIFY_FAILED?
@ -196,13 +217,11 @@ host.
.. seealso::
:doc:`index`
The documentation index
:doc:`windows`
:ref:`windows`
The Windows documentation index
:doc:`playbooks`
:ref:`about_playbooks`
An introduction to playbooks
:doc:`playbooks_best_practices`
:ref:`playbooks_best_practices`
Best practices advice
`User Mailing List <https://groups.google.com/group/ansible-project>`_
Have a question? Stop by the google group!

116
docs/docsite/rst/user_guide/windows_setup.rst
View file

@ -1,8 +1,10 @@
.. _windows_setup:
Setting up a Windows Host
=========================
This document discusses the setup that is required before Ansible can communicate with a Microsoft Windows host.
.. contents:: Topics
.. contents::
:local:
Host Requirements
@ -455,13 +457,117 @@ Sometimes an installer may restart the WinRM or HTTP service and cause this erro
best way to deal with this is to use ``win_psexec`` from another
Windows host.
Windows SSH Setup
`````````````````
Ansible 2.8 has added experimental support for using SSH to connect to a
Windows host.
.. warning::
Use this feature at your own risk!
Using SSH with Windows is experimental, the implementation may make
backwards incompatible changes in feature releases. The server side
components can be unreliable depending on the version that is installed.
Installing Win32-OpenSSH
------------------------
The first step to using SSH with Windows is to install the `Win32-OpenSSH <https://github.com/PowerShell/Win32-OpenSSH>`_
service on the Windows host. Microsoft offers a way to install ``Win32-OpenSSH`` through a Windows
capability but currently the version that is installed through this process is
too old to work with Ansible. To install ``Win32-OpenSSH`` for use with
Ansible, select one of these three installation options:
* Manually install the service, following the `install instructions <https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH>`_
from Microsoft.
* Use ``win_chocolatey`` to install the service::
- name: install the Win32-OpenSSH service
win_chocolatey:
name: openssh
package_params: /SSHServerFeature
state: present
* Use an existing Ansible Galaxy role like `jborean93.win_openssh <https://galaxy.ansible.com/jborean93/win_openssh>`_::
# Make sure the role has been downloaded first
ansible-galaxy install jborean93.win_openssh
# main.yml
- name: install Win32-OpenSSH service
hosts: windows
gather_facts: no
roles:
- role: jborean93.win_openssh
opt_openssh_setup_service: True
.. note:: ``Win32-OpenSSH`` is still a beta product and is constantly
being updated to include new features and bugfixes. If you are using SSH as
a connection option for Windows, it is highly recommend you install the
latest release from one of the 3 methods above.
Configuring the Win32-OpenSSH shell
-----------------------------------
By default ``Win32-OpenSSH`` will use ``cmd.exe`` as a shell. To configure a
different shell, use an Ansible task to define the registry setting::
- name: set the default shell to PowerShell
win_regedit:
path: HKLM:\SOFTWARE\OpenSSH
name: DefaultShell
data: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
type: string
state: present
# Or revert the settings back to the default, cmd
- name: set the default shell to cmd
win_regedit:
path: HKLM:\SOFTWARE\OpenSSH
name: DefaultShell
state: absent
Win32-OpenSSH Authentication
----------------------------
Win32-OpenSSH authentication with Windows is similar to SSH
authentication on Unix/Linux hosts. You can use a plaintext password or
SSH public key authentication, add public keys to an ``authorized_key`` file
in the ``.ssh`` folder of the user's profile directory, and configure the
service using the ``sshd_config`` file used by the SSH service as you would on
a Unix/Linux host.
When using SSH key authentication with Ansible, the remote session won't have access to the
user's credentials and will fail when attempting to access a network resource.
This is also known as the double-hop or credential delegation issue. There are
two ways to work around this issue:
* Use plaintext password auth by setting ``ansible_password``
* Use ``become`` on the task with the credentials of the user that needs access to the remote resource
Configuring Ansible for SSH on Windows
--------------------------------------
To configure Ansible to use SSH for Windows hosts, you must set two connection variables:
* set ``ansible_connection`` to ``ssh``
* set ``ansible_shell_type`` to ``cmd`` or ``powershell``
The ``ansible_shell_type`` variable should reflect the ``DefaultShell``
configured on the Windows host. Set to ``cmd`` for the default shell or set to
``powershell`` if the ``DefaultShell`` has been changed to PowerShell.
Known issues with SSH on Windows
--------------------------------
Using SSH with Windows is experimental, and we expect to uncover more issues.
Here are the known ones:
* Win32-OpenSSH versions older than ``v7.9.0.0p1-Beta`` do not work when ``powershell`` is the shell type
* While SCP should work, SFTP is the recommended SSH file transfer mechanism to use when copying or fetching a file
.. seealso::
:doc:`index`
The documentation index
:doc:`playbooks`
:ref:`about_playbooks`
An introduction to playbooks
:doc:`playbooks_best_practices`
:ref:`playbooks_best_practices`
Best practices advice
:ref:`List of Windows Modules <windows_modules>`
Windows specific module list, all implemented in PowerShell

2
lib/ansible/plugins/action/__init__.py
View file

@ -461,7 +461,7 @@ class ActionBase(with_metaclass(ABCMeta, object)):
if remote_user is None:
remote_user = self._get_remote_user()
if self._connection._shell.SHELL_FAMILY == 'powershell':
if getattr(self._connection._shell, "_IS_WINDOWS", False):
# This won't work on Powershell as-is, so we'll just completely skip until
# we have a need for it, at which point we'll have to do something different.
return remote_paths

6
lib/ansible/plugins/action/script.py
View file

@ -65,11 +65,11 @@ class ActionModule(ActionBase):
chdir = self._task.args.get('chdir')
if chdir:
# Powershell is the only Windows-path aware shell
if self._connection._shell.SHELL_FAMILY == 'powershell' and \
if getattr(self._connection._shell, "_IS_WINDOWS", False) and \
not self.windows_absolute_path_detection.match(chdir):
raise AnsibleActionFail('chdir %s must be an absolute path for a Windows remote node' % chdir)
# Every other shell is unix-path-aware.
if self._connection._shell.SHELL_FAMILY != 'powershell' and not chdir.startswith('/'):
if not getattr(self._connection._shell, "_IS_WINDOWS", False) and not chdir.startswith('/'):
raise AnsibleActionFail('chdir %s must be an absolute path for a Unix-aware remote node' % chdir)
# Split out the script as the first item in raw_params using
@ -126,7 +126,7 @@ class ActionModule(ActionBase):
exec_data = None
# PowerShell runs the script in a special wrapper to enable things
# like become and environment args
if self._connection._shell.SHELL_FAMILY == "powershell":
if getattr(self._connection._shell, "_IS_WINDOWS", False):
# FUTURE: use a more public method to get the exec payload
pc = self._play_context
exec_data = ps_manifest._create_powershell_wrapper(

2
lib/ansible/plugins/action/wait_for_connection.py
View file

@ -86,7 +86,7 @@ class ActionModule(ActionBase):
pass
# Use win_ping on winrm/powershell, else use ping
if hasattr(self._connection, '_shell_type') and self._connection._shell_type == 'powershell':
if getattr(self._connection._shell, "_IS_WINDOWS", False):
ping_result = self._execute_module(module_name='win_ping', module_args=dict(), task_vars=task_vars)
else:
ping_result = self._execute_module(module_name='ping', module_args=dict(), task_vars=task_vars)

1
lib/ansible/plugins/connection/__init__.py
View file

@ -8,6 +8,7 @@ __metaclass__ = type
import fcntl
import os
import shlex
from abc import abstractmethod, abstractproperty
from functools import wraps

61
lib/ansible/plugins/connection/ssh.py
View file

@ -276,6 +276,7 @@ import fcntl
import hashlib
import os
import pty
import re
import subprocess
import time
@ -294,6 +295,7 @@ from ansible.module_utils.six.moves import shlex_quote
from ansible.module_utils._text import to_bytes, to_native, to_text
from ansible.module_utils.parsing.convert_bool import BOOLEANS, boolean
from ansible.plugins.connection import ConnectionBase, BUFSIZE
from ansible.plugins.shell.powershell import _parse_clixml
from ansible.utils.display import Display
from ansible.utils.path import unfrackpath, makedirs_safe
@ -453,6 +455,15 @@ class Connection(ConnectionBase):
self.control_path = C.ANSIBLE_SSH_CONTROL_PATH
self.control_path_dir = C.ANSIBLE_SSH_CONTROL_PATH_DIR
# Windows operates differently from a POSIX connection/shell plugin,
# we need to set various properties to ensure SSH on Windows continues
# to work
if getattr(self._shell, "_IS_WINDOWS", False):
self.has_native_async = True
self.always_pipeline_modules = True
self.module_implementation_preferences = ('.ps1', '.exe', '')
self.allow_executable = False
# The connection is created by running ssh/scp/sftp from the exec_command,
# put_file, and fetch_file methods, so we don't need to do any connection
# management here.
@ -742,6 +753,7 @@ class Connection(ConnectionBase):
Starts the command and communicates with it until it ends.
'''
# We don't use _shell.quote as this is run on the controller and independent from the shell plugin chosen
display_cmd = list(map(shlex_quote, map(to_text, cmd)))
display.vvv(u'SSH: EXEC {0}'.format(u' '.join(display_cmd)), host=self.host)
@ -1030,6 +1042,12 @@ class Connection(ConnectionBase):
# accept them for hostnames and IPv4 addresses too.
host = '[%s]' % self.host
smart_methods = ['sftp', 'scp', 'piped']
# Windows does not support dd so we cannot use the piped method
if getattr(self._shell, "_IS_WINDOWS", False):
smart_methods.remove('piped')
# Transfer methods to try
methods = []
@ -1039,7 +1057,7 @@ class Connection(ConnectionBase):
if not (ssh_transfer_method in ('smart', 'sftp', 'scp', 'piped')):
raise AnsibleOptionsError('transfer_method needs to be one of [smart|sftp|scp|piped]')
if ssh_transfer_method == 'smart':
methods = ['sftp', 'scp', 'piped']
methods = smart_methods
else:
methods = [ssh_transfer_method]
else:
@ -1052,7 +1070,7 @@ class Connection(ConnectionBase):
elif scp_if_ssh != 'smart':
raise AnsibleOptionsError('scp_if_ssh needs to be one of [smart|True|False]')
if scp_if_ssh == 'smart':
methods = ['sftp', 'scp', 'piped']
methods = smart_methods
elif scp_if_ssh is True:
methods = ['scp']
else:
@ -1067,10 +1085,11 @@ class Connection(ConnectionBase):
(returncode, stdout, stderr) = self._bare_run(cmd, in_data, checkrc=False)
elif method == 'scp':
scp = self.get_option('scp_executable')
if sftp_action == 'get':
cmd = self._build_command(scp, u'{0}:{1}'.format(host, shlex_quote(in_path)), out_path)
cmd = self._build_command(scp, u'{0}:{1}'.format(host, self._shell.quote(in_path)), out_path)
else:
cmd = self._build_command(scp, in_path, u'{0}:{1}'.format(host, shlex_quote(out_path)))
cmd = self._build_command(scp, in_path, u'{0}:{1}'.format(host, self._shell.quote(out_path)))
in_data = None
(returncode, stdout, stderr) = self._bare_run(cmd, in_data, checkrc=False)
elif method == 'piped':
@ -1105,6 +1124,16 @@ class Connection(ConnectionBase):
raise AnsibleError("failed to transfer file to %s %s:\n%s\n%s" %
(to_native(in_path), to_native(out_path), to_native(stdout), to_native(stderr)))
def _escape_win_path(self, path):
""" converts a Windows path to one that's supported by SFTP and SCP """
# If using a root path then we need to start with /
prefix = ""
if re.match(r'^\w{1}:', path):
prefix = "/"
# Convert all '\' to '/'
return "%s%s" % (prefix, path.replace("\\", "/"))
#
# Main public methods
#
@ -1115,6 +1144,18 @@ class Connection(ConnectionBase):
display.vvv(u"ESTABLISH SSH CONNECTION FOR USER: {0}".format(self._play_context.remote_user), host=self._play_context.remote_addr)
if getattr(self._shell, "_IS_WINDOWS", False):
# Become method 'runas' is done in the wrapper that is executed,
# need to disable sudoable so the bare_run is not waiting for a
# prompt that will not occur
sudoable = False
# Make sure our first command is to set the console encoding to
# utf-8, this must be done via chcp to get utf-8 (65001)
cmd_parts = ["chcp.com", "65001", self._shell._SHELL_REDIRECT_ALLNULL, self._shell._SHELL_AND]
cmd_parts.extend(self._shell._encode_script(cmd, as_list=True, strict_mode=False, preserve_rc=False))
cmd = ' '.join(cmd_parts)
# we can only use tty when we are not pipelining the modules. piping
# data into /usr/bin/python inside a tty automatically invokes the
# python interactive-mode but the modules are not compatible with the
@ -1134,6 +1175,10 @@ class Connection(ConnectionBase):
cmd = self._build_command(*args)
(returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
# When running on Windows, stderr may contain CLIXML encoded output
if getattr(self._shell, "_IS_WINDOWS", False) and stderr.startswith(b"#< CLIXML"):
stderr = _parse_clixml(stderr)
return (returncode, stdout, stderr)
def put_file(self, in_path, out_path):
@ -1145,6 +1190,9 @@ class Connection(ConnectionBase):
if not os.path.exists(to_bytes(in_path, errors='surrogate_or_strict')):
raise AnsibleFileNotFound("file or module does not exist: {0}".format(to_native(in_path)))
if getattr(self._shell, "_IS_WINDOWS", False):
out_path = self._escape_win_path(out_path)
return self._file_transport_command(in_path, out_path, 'put')
def fetch_file(self, in_path, out_path):
@ -1153,6 +1201,11 @@ class Connection(ConnectionBase):
super(Connection, self).fetch_file(in_path, out_path)
display.vvv(u"FETCH {0} TO {1}".format(in_path, out_path), host=self.host)
# need to add / if path is rooted
if getattr(self._shell, "_IS_WINDOWS", False):
in_path = self._escape_win_path(in_path)
return self._file_transport_command(in_path, out_path, 'get')
def reset(self):

19
lib/ansible/plugins/connection/winrm.py
View file

@ -104,7 +104,6 @@ import traceback
import json
import tempfile
import subprocess
import xml.etree.ElementTree as ET
HAVE_KERBEROS = False
try:
@ -122,6 +121,7 @@ from ansible.module_utils.six.moves.urllib.parse import urlunsplit
from ansible.module_utils._text import to_bytes, to_native, to_text
from ansible.module_utils.six import binary_type, PY3
from ansible.plugins.connection import ConnectionBase
from ansible.plugins.shell.powershell import _parse_clixml
from ansible.utils.hashing import secure_hash
from ansible.utils.path import makedirs_safe
from ansible.utils.display import Display
@ -538,28 +538,15 @@ class Connection(ConnectionBase):
result.std_err = to_bytes(result.std_err)
# parse just stderr from CLIXML output
if self.is_clixml(result.std_err):
if result.std_err.startswith(b"#< CLIXML"):
try:
result.std_err = self.parse_clixml_stream(result.std_err)
result.std_err = _parse_clixml(result.std_err)
except Exception:
# unsure if we're guaranteed a valid xml doc- use raw output in case of error
pass
return (result.status_code, result.std_out, result.std_err)
def is_clixml(self, value):
return value.startswith(b"#< CLIXML\r\n")
# hacky way to get just stdout- not always sure of doc framing here, so use with care
def parse_clixml_stream(self, clixml_doc, stream_name='Error'):
clixml = ET.fromstring(clixml_doc.split(b"\r\n", 1)[-1])
namespace_match = re.match(r'{(.*)}', clixml.tag)
namespace = "{%s}" % namespace_match.group(1) if namespace_match else ""
strings = clixml.findall("./%sS" % namespace)
lines = [e.text.replace('_x000D__x000A_', '') for e in strings if e.attrib.get('S') == stream_name]
return to_bytes('\r\n'.join(lines))
# FUTURE: determine buffer size at runtime via remote winrm config?
def _put_file_stdin_iterator(self, in_path, out_path, buffer_size=250000):
in_size = os.path.getsize(to_bytes(in_path, errors='surrogate_or_strict'))

47
lib/ansible/plugins/doc_fragments/shell_windows.py Normal file
View file

@ -0,0 +1,47 @@
# Copyright (c) 2019 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
class ModuleDocFragment(object):
# Windows shell documentation fragment
# FIXME: set_module_language don't belong here but must be set so they don't fail when someone
# get_option('set_module_language') on this plugin
DOCUMENTATION = """
options:
async_dir:
description:
- Directory in which ansible will keep async job information.
- Before Ansible 2.8, this was set to C(remote_tmp + "\\.ansible_async").
default: '%USERPROFILE%\\.ansible_async'
ini:
- section: powershell
key: async_dir
vars:
- name: ansible_async_dir
version_added: '2.8'
remote_tmp:
description:
- Temporary directory to use on targets when copying files to the host.
default: '%TEMP%'
ini:
- section: powershell
key: remote_tmp
vars:
- name: ansible_remote_tmp
set_module_language:
description:
- Controls if we set the locale for moduels when executing on the
target.
- Windows only supports C(no) as an option.
type: bool
default: 'no'
choices:
- 'no'
environment:
description:
- Dictionary of environment variables and their values to use when
executing commands.
type: dict
default: {}
"""

4
lib/ansible/plugins/shell/__init__.py
View file

@ -221,3 +221,7 @@ class ShellBase(AnsiblePlugin):
def wrap_for_exec(self, cmd):
"""wrap script execution with any necessary decoration (eg '&' for quoted powershell script paths)"""
return cmd
def quote(self, cmd):
"""Returns a shell-escaped string that can be safely used as one token in a shell command line"""
return shlex_quote(cmd)

57
lib/ansible/plugins/shell/cmd.py Normal file
View file

@ -0,0 +1,57 @@
# Copyright (c) 2019 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = '''
name: cmd
plugin_type: shell
version_added: '2.8'
short_description: Windows Command Prompt
description:
- Used with the 'ssh' connection plugin and no C(DefaultShell) has been set on the Windows host.
extends_documentation_fragment:
- shell_windows
'''
import re
from ansible.plugins.shell.powershell import ShellModule as PSShellModule
# these are the metachars that have a special meaning in cmd that we want to escape when quoting
_find_unsafe = re.compile(r'[\s\(\)\%\!^\"\<\>\&\|]').search
class ShellModule(PSShellModule):
# Common shell filenames that this plugin handles
COMPATIBLE_SHELLS = frozenset()
# Family of shells this has. Must match the filename without extension
SHELL_FAMILY = 'cmd'
_SHELL_REDIRECT_ALLNULL = '>nul 2>&1'
_SHELL_AND = '&&'
# Used by various parts of Ansible to do Windows specific changes
_IS_WINDOWS = True
def quote(self, s):
# cmd does not support single quotes that the shlex_quote uses. We need to override the quoting behaviour to
# better match cmd.exe.
# https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/
# Return an empty argument
if not s:
return '""'
if _find_unsafe(s) is None:
return s
# Escape the metachars as we are quoting the string to stop cmd from interpreting that metachar. For example
# 'file &whoami.exe' would result in 'file $(whoami.exe)' instead of the literal string
# https://stackoverflow.com/questions/3411771/multiple-character-replace-with-python
for c in '^()%!"<>&|': # '^' must be the first char that we scan and replace
if c in s:
s = s.replace(c, "^" + c)
return '^"' + s + '^"'

71
lib/ansible/plugins/shell/powershell.py
View file

@ -7,58 +7,24 @@ __metaclass__ = type
DOCUMENTATION = '''
name: powershell
plugin_type: shell
version_added: ""
short_description: Windows Powershell
version_added: historical
short_description: Windows PowerShell
description:
- The only option when using 'winrm' as a connection plugin
options:
async_dir:
description:
- Directory in which ansible will keep async job information.
- Before Ansible 2.8, this was set to C(remote_tmp + "\\.ansible_async").
default: '%USERPROFILE%\\.ansible_async'
ini:
- section: powershell
key: async_dir
vars:
- name: ansible_async_dir
version_added: '2.8'
remote_tmp:
description:
- Temporary directory to use on targets when copying files to the host.
default: '%TEMP%'
ini:
- section: powershell
key: remote_tmp
vars:
- name: ansible_remote_tmp
set_module_language:
description:
- Controls if we set the locale for moduels when executing on the
target.
- Windows only supports C(no) as an option.
type: bool
default: 'no'
choices:
- 'no'
environment:
description:
- Dictionary of environment variables and their values to use when
executing commands.
type: dict
default: {}
- The only option when using 'winrm' or 'psrp' as a connection plugin.
- Can also be used when using 'ssh' as a connection plugin and the C(DefaultShell) has been configured to PowerShell.
extends_documentation_fragment:
- shell_windows
'''
# FIXME: admin_users and set_module_language don't belong here but must be set
# so they don't failk when someone get_option('admin_users') on this plugin
import base64
import os
import re
import shlex
import pkgutil
import xml.etree.ElementTree as ET
from ansible.errors import AnsibleError
from ansible.module_utils._text import to_text
from ansible.module_utils._text import to_bytes, to_text
from ansible.plugins.shell import ShellBase
@ -71,6 +37,21 @@ if _powershell_version:
_common_args = ['PowerShell', '-Version', _powershell_version] + _common_args[1:]
def _parse_clixml(data, stream="Error"):
"""
Takes a byte string like '#< CLIXML\r\n<Objs...' and extracts the stream
message encoded in the XML data. CLIXML is used by PowerShell to encode
multiple objects in stderr.
"""
clixml = ET.fromstring(data.split(b"\r\n", 1)[-1])
namespace_match = re.match(r'{(.*)}', clixml.tag)
namespace = "{%s}" % namespace_match.group(1) if namespace_match else ""
strings = clixml.findall("./%sS" % namespace)
lines = [e.text.replace('_x000D__x000A_', '') for e in strings if e.attrib.get('S') == stream]
return to_bytes('\r\n'.join(lines))
class ShellModule(ShellBase):
# Common shell filenames that this plugin handles
@ -80,6 +61,12 @@ class ShellModule(ShellBase):
# Family of shells this has. Must match the filename without extension
SHELL_FAMILY = 'powershell'
_SHELL_REDIRECT_ALLNULL = '> $null'
_SHELL_AND = ';'
# Used by various parts of Ansible to do Windows specific changes
_IS_WINDOWS = True
env = dict()
# We're being overly cautious about which keys to accept (more so than

6
test/integration/targets/connection_windows_ssh/aliases Normal file
View file

@ -0,0 +1,6 @@
windows
shippable/windows/group1
shippable/windows/smoketest
skip/windows/2008 # Windows Server 2008 does not support Win32-OpenSSH
needs/target/connection
needs/target/setup_remote_tmp_dir

54
test/integration/targets/connection_windows_ssh/runme.sh Executable file
View file

@ -0,0 +1,54 @@
#!/usr/bin/env bash
set -eux
# We need to run these tests with both the powershell and cmd shell type
### cmd tests - no DefaultShell set ###
ansible -i ../../inventory.winrm localhost \
-m template \
-a "src=test_connection.inventory.j2 dest=~/ansible_testing/test_connection.inventory" \
-e "test_shell_type=cmd" \
"$@"
# https://github.com/PowerShell/Win32-OpenSSH/wiki/DefaultShell
ansible -i ../../inventory.winrm windows \
-m win_regedit \
-a "path=HKLM:\\\\SOFTWARE\\\\OpenSSH name=DefaultShell state=absent" \
"$@"
# Need to flush the connection to ensure we get a new shell for the next tests
ansible -i ~/ansible_testing/test_connection.inventory windows-ssh \
-m meta -a "reset_connection" \
"$@"
# sftp
./windows.sh "$@"
# scp
ANSIBLE_SCP_IF_SSH=true ./windows.sh "$@"
# other tests not part of the generic connection test framework
ansible-playbook -i ~/ansible_testing/test_connection.inventory tests.yml \
"$@"
### powershell tests - explicit DefaultShell set ###
# we do this last as the default shell on our CI instances is set to PowerShell
ansible -i ../../inventory.winrm localhost \
-m template \
-a "src=test_connection.inventory.j2 dest=~/ansible_testing/test_connection.inventory" \
-e "test_shell_type=powershell" \
"$@"
# ensure the default shell is set to PowerShell
ansible -i ../../inventory.winrm windows \
-m win_regedit \
-a "path=HKLM:\\\\SOFTWARE\\\\OpenSSH name=DefaultShell data=C:\\\\Windows\\\\System32\\\\WindowsPowerShell\\\\v1.0\\\\powershell.exe" \
"$@"
ansible -i ~/ansible_testing/test_connection.inventory windows-ssh \
-m meta -a "reset_connection" \
"$@"
./windows.sh "$@"
ANSIBLE_SCP_IF_SSH=true ./windows.sh "$@"
ansible-playbook -i ~/ansible_testing/test_connection.inventory tests.yml \
"$@"

12
test/integration/targets/connection_windows_ssh/test_connection.inventory.j2 Normal file
View file

@ -0,0 +1,12 @@
[windows-ssh]
{% for host in vars.groups.winrm %}
{{ host }} ansible_host={{ hostvars[host]['ansible_host'] }} ansible_user={{ hostvars[host]['ansible_user'] }}{{ ' ansible_ssh_private_key_file=' ~ hostvars[host]['ansible_ssh_private_key_file'] if (hostvars[host]['ansible_ssh_private_key_file']|default()) else '' }}
{% endfor %}
[windows-ssh:vars]
ansible_shell_type={{ test_shell_type }}
ansible_connection=ssh
ansible_port=22
# used to preserve the existing environment and not touch existing files
ansible_ssh_extra_args="-o UserKnownHostsFile=/dev/null"
ansible_ssh_host_key_checking=False

32
test/integration/targets/connection_windows_ssh/tests.yml Normal file
View file

@ -0,0 +1,32 @@
---
- name: test out Windows SSH specific tests
hosts: windows-ssh
serial: 1
gather_facts: no
tasks:
- name: test out become with Windows SSH
win_whoami:
register: win_ssh_become
become: yes
become_method: runas
become_user: SYSTEM
- name: assert test out become with Windows SSH
assert:
that:
- win_ssh_become.account.sid == "S-1-5-18"
- name: test out async with Windows SSH
win_shell: Write-Host café
async: 20
poll: 3
register: win_ssh_async
- name: assert test out async with Windows SSH
assert:
that:
- win_ssh_async is changed
- win_ssh_async.rc == 0
- win_ssh_async.stdout == "café\n"
- win_ssh_async.stderr == ""

41
test/integration/targets/connection_windows_ssh/tests_fetch.yml Normal file
View file

@ -0,0 +1,41 @@
# This must be a play as we need to invoke it with the ANSIBLE_SCP_IF_SSH env
# to control the mechanism used. Unfortunately while ansible_scp_if_ssh is
# documented, it isn't actually used hence the separate invocation
---
- name: further fetch tests with metachar characters in filename
hosts: windows-ssh
force_handlers: yes
serial: 1
gather_facts: no
tasks:
- name: setup remote tmp dir
import_role:
name: ../../setup_remote_tmp_dir
- name: create remote file with metachar in name
win_copy:
content: some content
dest: '{{ remote_tmp_dir }}\file ^with &whoami'
- name: test fetch against a file with cmd metacharacters
block:
- name: fetch file with metachar in name
fetch:
src: '{{ remote_tmp_dir }}\file ^with &whoami'
dest: ansible-test.txt
flat: yes
register: fetch_res
- name: assert fetch file with metachar in name
assert:
that:
- fetch_res is changed
- fetch_res.checksum == '94e66df8cd09d410c62d9e0dc59d3a884e458e05'
always:
- name: remove local copy of file
file:
path: ansible-test.txt
state: absent
delegate_to: localhost

25
test/integration/targets/connection_windows_ssh/windows.sh Executable file
View file

@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -eux
cd ../connection
# A recent patch to OpenSSH causes a validation error when running through Ansible. It seems like if the path is quoted
# then it will fail with 'protocol error: filename does not match request'. We currently ignore this by setting
# 'ansible_scp_extra_args=-T' to ignore this check but this should be removed once that bug is fixed and our test
# container has been updated.
# https://unix.stackexchange.com/questions/499958/why-does-scps-strict-filename-checking-reject-quoted-last-component-but-not-oth
# https://github.com/openssh/openssh-portable/commit/391ffc4b9d31fa1f4ad566499fef9176ff8a07dc
INVENTORY=~/ansible_testing/test_connection.inventory ./test.sh \
-e target_hosts=windows-ssh \
-e action_prefix=win_ \
-e local_tmp=/tmp/ansible-local \
-e remote_tmp=c:/windows/temp/ansible-remote \
-e ansible_scp_extra_args=-T \
"$@"
cd ../connection_windows_ssh
ansible-playbook -i ~/ansible_testing/test_connection.inventory tests_fetch.yml \
-e ansible_scp_extra_args=-T \
"$@"

6
test/runner/lib/classification.py
View file

@ -474,6 +474,11 @@ class PathMapper(object):
if integration_name not in self.integration_targets_by_name:
integration_name = None
windows_integration_name = 'connection_windows_%s' % name
if windows_integration_name not in self.integration_targets_by_name:
windows_integration_name = None
# entire integration test commands depend on these connection plugins
if name in ['winrm', 'psrp']:
@ -506,6 +511,7 @@ class PathMapper(object):
return {
'integration': integration_name,
'windows-integration': windows_integration_name,
'units': units_path,
}

4
test/runner/lib/executor.py
View file

@ -694,6 +694,10 @@ def windows_inventory(remotes):
ansible_port=remote.connection.port,
)
# used for the connection_windows_ssh test target
if remote.ssh_key:
options["ansible_ssh_private_key_file"] = os.path.abspath(remote.ssh_key.key)
hosts.append(
'%s %s' % (
remote.name.replace('/', '_'),

16
test/units/plugins/shell/test_cmd.py Normal file
View file

@ -0,0 +1,16 @@
import pytest
from ansible.plugins.shell.cmd import ShellModule
@pytest.mark.parametrize('s, expected', [
['arg1', 'arg1'],
[None, '""'],
['arg1 and 2', '^"arg1 and 2^"'],
['malicious argument\\"&whoami', '^"malicious argument\\^"^&whoami^"'],
['C:\\temp\\some ^%file% > nul', '^"C:\\temp\\some ^^^%file^% ^> nul^"']
])
def test_quote_args(s, expected):
cmd = ShellModule()
actual = cmd.quote(s)
assert actual == expected

53
test/units/plugins/shell/test_powershell.py Normal file
View file

@ -0,0 +1,53 @@
from ansible.plugins.shell.powershell import _parse_clixml
def test_parse_clixml_empty():
empty = b'#< CLIXML\r\n<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"></Objs>'
expected = b''
actual = _parse_clixml(empty)
assert actual == expected
def test_parse_clixml_with_progress():
progress = b'#< CLIXML\r\n<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">' \
b'<Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS>' \
b'<I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil />' \
b'<PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>'
expected = b''
actual = _parse_clixml(progress)
assert actual == expected
def test_parse_clixml_single_stream():
single_stream = b'#< CLIXML\r\n<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">' \
b'<S S="Error">fake : The term \'fake\' is not recognized as the name of a cmdlet. Check _x000D__x000A_</S>' \
b'<S S="Error">the spelling of the name, or if a path was included._x000D__x000A_</S>' \
b'<S S="Error">At line:1 char:1_x000D__x000A_</S>' \
b'<S S="Error">+ fake cmdlet_x000D__x000A_</S><S S="Error">+ ~~~~_x000D__x000A_</S>' \
b'<S S="Error"> + CategoryInfo : ObjectNotFound: (fake:String) [], CommandNotFoundException_x000D__x000A_</S>' \
b'<S S="Error"> + FullyQualifiedErrorId : CommandNotFoundException_x000D__x000A_</S><S S="Error"> _x000D__x000A_</S>' \
b'</Objs>'
expected = b"fake : The term 'fake' is not recognized as the name of a cmdlet. Check \r\n" \
b"the spelling of the name, or if a path was included.\r\n" \
b"At line:1 char:1\r\n" \
b"+ fake cmdlet\r\n" \
b"+ ~~~~\r\n" \
b" + CategoryInfo : ObjectNotFound: (fake:String) [], CommandNotFoundException\r\n" \
b" + FullyQualifiedErrorId : CommandNotFoundException\r\n "
actual = _parse_clixml(single_stream)
assert actual == expected
def test_parse_clixml_multiple_streams():
multiple_stream = b'#< CLIXML\r\n<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">' \
b'<S S="Error">fake : The term \'fake\' is not recognized as the name of a cmdlet. Check _x000D__x000A_</S>' \
b'<S S="Error">the spelling of the name, or if a path was included._x000D__x000A_</S>' \
b'<S S="Error">At line:1 char:1_x000D__x000A_</S>' \
b'<S S="Error">+ fake cmdlet_x000D__x000A_</S><S S="Error">+ ~~~~_x000D__x000A_</S>' \
b'<S S="Error"> + CategoryInfo : ObjectNotFound: (fake:String) [], CommandNotFoundException_x000D__x000A_</S>' \
b'<S S="Error"> + FullyQualifiedErrorId : CommandNotFoundException_x000D__x000A_</S><S S="Error"> _x000D__x000A_</S>' \
b'<S S="Info">hi info</S>' \
b'</Objs>'
expected = b"hi info"
actual = _parse_clixml(multiple_stream, stream="Info")
assert actual == expected