diff --git a/docsite/rst/developing_modules_python3.rst b/docsite/rst/developing_modules_python3.rst new file mode 100644 index 0000000000..c9d246d086 --- /dev/null +++ b/docsite/rst/developing_modules_python3.rst @@ -0,0 +1,135 @@ +=========================== +Porting Modules to Python 3 +=========================== + +Ansible modules are not the usual Python-3 porting exercise. There are two +factors that make it harder to port them than most code: + +1. Many modules need to run on Python-2.4 in addition to Python-3. +2. A lot of mocking has to go into unittesting a Python-3 module. So it's + harder to test that your porting has fixed everything or to make sure that + later commits haven't regressed. + +Which version of Python-3.x and which version of Python-2.x are our minimums? +============================================================================= + +The short answer is Python-3.4 and Python-2.4 but please read on for more +information. + +For Python-3 we are currently using Python-3.4 as a minimum. However, no long +term supported Linux distributions currently ship with Python-3. When that +occurs, we will probably take that as our minimum Python-3 version rather than +Python-3.4. Thus far, Python-3 has been adding small changes that make it +more compatible with Python-2 in its newer versions (For instance, Python-3.5 +added the ability to use percent-formatted byte strings.) so it should be more +pleasant to use a newer version of Python-3 if it's available. At some point +this will change but we'll just have to cross that bridge when we get to it. + +For Python-2 the default is for modules to run on Python-2.4. This allows +users with older distributions that are stuck on Python-2.4 to manage their +machines. Modules are allowed to drop support for Python-2.4 when one of +their dependent libraries require a higher version of python. This is not an +invitation to add unnecessary dependent libraries in order to force your +module to be usable only with a newer version of Python. Instead it is an +acknowledgment that some libraries (for instance, boto3 and docker-py) will +only function with newer Python. + +.. note:: When will we drop support for Python-2.4? + + The only long term supported distro that we know of with Python-2.4 is + RHEL5 (and its rebuilds like CentOS5) which is supported until April of + 2017. We will likely end our support for Python-2.4 in modules in an + Ansible release around that time. We know of no long term supported + distributions with Python-2.5 so the new minimum Python-2 version will + likely be Python-2.6. This will let us take advantage of the + forwards-compat features of Python-2.6 so porting and maintainance of + Python-2/Python-3 code will be easier after that. + +Supporting only Python-2 or only Python-3 +========================================= + +Sometimes a module's dependent libraries only run on Python-2 or only run on +Python-3. We do not yet have a strategy for these modules but we'll need to +come up with one. I see three possibilities: + +1. We treat these libraries like any other libraries that may not be installed + on the system. When we import them we check if the import was successful. + If so, then we continue. If not we return an error about the library being + missing. Users will have to find out that the library is unavailable on + their version of Python either by searching for the library on their own or + reading the requirements section in :command:`ansible-doc`. + +2. The shebang line is the only metadata that Ansible extracts from a module + so we may end up using that to specify what we mean. Something like + ``#!/usr/bin/python`` means the module will run on both Python-2 and + Python-3, ``#!/usr/bin/python2`` means the module will only run on + Python-2, and ``#!/usr/bin/python3`` means the module will only run on + Python-3. Ansible's code will need to be modified to accommodate this. + For :command:`python2`, if ``ansible_python2_interpreter`` is not set, it + will have to fallback to `` ansible_python_interpreter`` and if that's not + set, fallback to ``/usr/bin/python``. For :command:`python3`, Ansible + will have to first try ``ansible_python3_interpreter`` and then fallback to + ``/usr/bin/python3`` as normal. + +3. We add a way for Ansible to retrieve metadata about modules. The metadata + will include the version of Python that is required. + +Methods 2 and 3 will both require that we modify modules or otherwise add this +additional information somewhere. + +Tips, tricks, and idioms to adopt +================================= + +Exceptions +---------- + +In code which already needs Python-2.6+ (For instance, because a library it +depends on only runs on Python >= 2.6) it is okay to port directly to the new +exception catching syntax:: + + try: + a = 2/0 + except ValueError as e: + module.fail_json(msg="Tried to divide by zero!") + +For modules which also run on Python-2.4, we have to use an uglier +construction to make this work under both Python-2.4 and Python-3:: + + from ansible.module_utils.pycompat import get_exception + [...] + + try: + a = 2/0 + except ValueError: + e = get_exception() + module.fail_json(msg="Tried to divide by zero!") + +Octal numbers +------------- + +In Python-2.4, octal literals are specified as ``0755``. In Python-3, that is +invalid and octals must be specified as ``0o755``. To bridge this gap, +modules should create their octals like this:: + + # Can't use 0755 on Python-3 and can't use 0o755 on Python-2.4 + EXECUTABLE_PERMS = int('0755', 8) + + +Compile Test +------------ + +We have travis compiling all modules with various versions of Python to check +that the modules conform to the syntax at those versions. When you've +ported a module so that its syntax works with Python-3, we need to modify +.travis.yml so that the module is included in the syntax check. Here's the +relevant section of .travis.yml:: + + script: + [...] + - python3.4 -m compileall -fq system/ping.py + - python3.5 -m compileall -fq system/ping.py + +At the moment this is a whitelist. Just add your newly ported module to that +line. Eventually, not compiling on Python-3 will be the exception. When that +occurs, we will move to a blacklist for listing which modules do not compile +under Python-3.