mirror of
https://github.com/ansible-collections/community.general.git
synced 2024-09-14 20:13:21 +02:00
Add PR testing guide (95% from James).
This commit is contained in:
Michael DeHaan
parent
2387da09a0
commit
c05a12472c
2 changed files with 171 additions and 0 deletions
|
|
@ -10,6 +10,7 @@ Learn how to build modules of your own in any language, and also how to extend A
|
|||
developing_inventory
|
||||
developing_modules
|
||||
developing_plugins
|
||||
developing_test_pr
|
||||
|
||||
Developers will also likely be interested in the fully-discoverable in :doc:`tower`. It's great for embedding Ansible in all manner of applications.
|
||||
|
||||
|
|
|
|||
170
docsite/rst/developing_test_pr.rst
Normal file
170
docsite/rst/developing_test_pr.rst
Normal file
|
|
@ -0,0 +1,170 @@
|
|||
Helping Testing PRs
|
||||
```````````````````
|
||||
|
||||
If you're a developer, one of the most valuable things you can do is look at the github issues list and help fix bugs. We almost always prioritize bug fixing over
|
||||
feature development, so clearing bugs out of the way is one of the best things you can do.
|
||||
|
||||
Even if you're not a developer, helping test pull requests for bug fixes and features is still immensely valuable.
|
||||
|
||||
This goes for testing new features as well as testing bugfixes.
|
||||
|
||||
In many cases, code should add tests that prove it works but that's not ALWAYS possible and tests are not always comprehensive, especially when a user doesn't have access
|
||||
to a wide variety of platforms, or that is using an API or web service.
|
||||
|
||||
In these cases, live testing against real equipment can be more valuable than automation that runs against simulated interfaces.
|
||||
In any case, things should always be tested manually the first time too.
|
||||
|
||||
Thankfully helping test ansible is pretty straightforward, assuming you are already used to how ansible works.
|
||||
|
||||
Get Started with A Source Checkout
|
||||
++++++++++++++++++++++++++++++++++
|
||||
|
||||
You can do this by checking out ansible, making a test branch off the main one, merging a GitHub issue, testing,
|
||||
and then commenting on that particular issue on GitHub. Here's how:
|
||||
|
||||
.. note::
|
||||
Testing source code from GitHub pull requests sent to us does have some inherent risk, as the source code
|
||||
sent may have mistakes or malicious code that could have a negative impact on your system. We recommend
|
||||
doing all testing on a virtual machine, whether a cloud instance, or locally. Some users like Vagrant
|
||||
or Docker for this, but they are optional. It is also useful to have virtual machines of different Linux or
|
||||
other flavors, since some features (apt vs. yum, for example) are specific to those OS versions.
|
||||
|
||||
First, you will need to configure your testing environment with the neccessary tools required to run our test
|
||||
suites. You will need at least::
|
||||
|
||||
git
|
||||
python-nosetests
|
||||
|
||||
Second, if you haven't already, clone the Ansible source code from GitHub::
|
||||
|
||||
git clone https://github.com/ansible/ansible.git
|
||||
cd ansible/
|
||||
|
||||
.. note::
|
||||
If you have previously forked the repository on GitHub, you could also clone it from there.
|
||||
|
||||
Activating The Source Checkout
|
||||
++++++++++++++++++++++++++++++
|
||||
|
||||
The Ansible source includes a script that allows you to use Ansible directly from source without requiring a
|
||||
full installation, that is frequently used by developers on Ansible.
|
||||
|
||||
Simply source it (to use the Linux/Unix terminology) to begin using it immediately::
|
||||
|
||||
source ./hacking/env-setup
|
||||
|
||||
This script modifies the PYTHONPATH enviromnent variables (along with a few other things), which will be temporarily
|
||||
set as long as your shell session is open.
|
||||
|
||||
If you'd like your testing environment to always use the latest source, you could call the command from startup scripts (for example,
|
||||
`.bash_profile`).
|
||||
|
||||
Finding A Pull Request and Checking It Out On A Branch
|
||||
++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
Next, find the pull request you'd like to test and make note of the line at the top which describes the source
|
||||
and destination repositories. It will look something like this::
|
||||
|
||||
Someuser wants to merge 1 commit into ansible:devel from someuser:feature_branch_name
|
||||
|
||||
.. note::
|
||||
It is important that the PR request target be ansible:devel, as we do not accept pull requests into any other branch.
|
||||
Dot releases are cherry-picked manually by ansible staff.
|
||||
|
||||
At the bottom of the GitHub page, there is a link that says "You can also merge this request on the command line". Click this link
|
||||
to expand the GitHub interface, and it will show instructions that look somewhat like this::
|
||||
|
||||
Step 1: From your project repository, check out a new branch and test the changes.
|
||||
|
||||
git checkout -b username-branchname branchname
|
||||
git pull git://github.com/username/branchname.git branchname
|
||||
|
||||
Do not follow step 2, as you don't want to merge the user features back into your branch.
|
||||
|
||||
.. note::
|
||||
If the GitHub user interface shows that the pull request will not merge cleanly, we do not recommend proceeding if you
|
||||
are not somewhat familiar with git and coding, as you will have to resolve a merge conflict. This is the responsibility of
|
||||
the original pull request contributor.
|
||||
|
||||
.. note::
|
||||
Some users do not create feature branches, which can cause problems when they have multiple, un-related commits in
|
||||
their version of `devel`. If the source looks like `someuser:devel`, make sure there is only one commit listed on
|
||||
the pull request.
|
||||
|
||||
For Those About To Test, We Salute You
|
||||
++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
At this point, you should be ready to begin testing!
|
||||
|
||||
If the PR is a bug-fix pull request, the first things to do are to run the suite of unit and integration tests, to ensure
|
||||
the pull request does not break current functionality::
|
||||
|
||||
# Unit Tests
|
||||
make tests
|
||||
|
||||
# Integration Tests
|
||||
cd test/integration
|
||||
make
|
||||
|
||||
.. note::
|
||||
Ansible does provide integration tests for cloud-based modules as well, however we do not recommend using them for some users
|
||||
due to the associated costs from the cloud providers. As such, typically it's better to run specific parts of the integration battery
|
||||
and skip these tests.
|
||||
|
||||
Integration tests aren't the end all beat all - in many cases what is fixed might not *HAVE* a test, so determining if it works means
|
||||
checking the functionality of the system and making sure it does what it said it would do.
|
||||
|
||||
Pull requests for bug-fixes should reference the bug issue number they are fixing.
|
||||
|
||||
We encourage users to provide playbook examples for bugs that show how to reproduce the error, and these playbooks should be used to verify the bugfix does resolve
|
||||
the issue if available. You may wish to also do your own review to poke the corners of the change.
|
||||
|
||||
Since some reproducers can be quite involved, you might wish to create a testing directory with the issue # as a sub-
|
||||
directory to keep things organized::
|
||||
|
||||
mkdir -p testing/XXXX # where XXXX is again the issue # for the original issue or PR
|
||||
cd testing/XXXX
|
||||
<create files or git clone example playbook repo>
|
||||
|
||||
While it should go without saying, be sure to read any playbooks before you run them. VMs help with running untrusted content greatly,
|
||||
though a playbook could still do something to your computing resources that you'd rather not like.
|
||||
|
||||
Once the files are in place, you can run the provided playbook (if there is one) to test the functionality::
|
||||
|
||||
ansible-playbook -vvv playbook_name.yml
|
||||
|
||||
If there's not a playbook, you may have to copy and paste playbook snippets or run a ad-hoc command that was pasted in.
|
||||
|
||||
Our issue template also included sections for "Expected Output" and "Actual Output", which should be used to gauge the output
|
||||
from the provided examples.
|
||||
|
||||
If the pull request resolves the issue, please leave a comment on the pull request, showing the following information:
|
||||
|
||||
* "Works for me!"
|
||||
* The output from `ansible --version`.
|
||||
|
||||
In some cases, you may wish to share playbook output from the test run as well.
|
||||
|
||||
Example!::
|
||||
|
||||
Works for me! Tested on `Ansible 1.7.1`. I verified this on CentOS 6.5 and also Ubuntu 14.04.
|
||||
|
||||
If the PR does not resolve the issue, or if you see any failures from the unit/integration tests, just include that output instead::
|
||||
|
||||
This doesn't work for me.
|
||||
|
||||
When I ran this my toaster started making loud noises!
|
||||
|
||||
Output from the toaster looked like this:
|
||||
|
||||
```
|
||||
BLARG
|
||||
StrackTrace
|
||||
RRRARRGGG
|
||||
```
|
||||
|
||||
We understand some users may be inexperienced with git, or other aspects of the above procedure, so feel free to stop by ansible-devel
|
||||
list for questions and we'd be happy to help answer them.
|
||||
|
||||
|
||||
|
||||
Loading…
Reference in a new issue