Matt Clay
5cb5228cde
Implement on-demand coverage on Shippable. ( #24535 )
2017-05-14 01:08:42 +08:00
Matt Clay
af5db3e4a9
Revert "Add support for on-demand coverage on Shippable."
...
This reverts commit 7b3f0b8870
.
2017-05-12 15:47:28 +08:00
Matt Clay
7b3f0b8870
Add support for on-demand coverage on Shippable.
...
Triggered by setting the COVERAGE environment var to a non-empty value.
Alternatively the string `ci_coverage` can be added to a commit message.
2017-05-12 14:55:48 +08:00
Matt Clay
fb9efd3c7e
Add --remote-terminate
option to ansible-test.
...
Set `--remote-terminate success` for osx tests.
2017-05-11 22:42:52 +08:00
René Moser
aaf4f04574
automated integration tests for cloudstack ( #20552 )
2017-05-09 11:32:11 +08:00
Matt Clay
17e07a27b2
Enable cloud tests for use with ansible-test.
2017-05-05 21:46:29 +08:00
John R Barker
ecbf8e933a
Docs how to test (2nd) ( #24094 )
...
* Big testing doc refactor
* Combine all the testing documentation in to one place to make it easier to find
* Convert everything to RST
* Create testing_network guide
* Create testing landing page
* For each section detail "how to run" and "how to extend testing"
* More examples
* Lots more detail
2017-04-28 09:08:26 +01:00
Matt Clay
0587d59266
Use Shippable image: drydock/u16pytall:master ( #23997 )
...
* Use Shippable image: drydock/u16pytall:master
* Do not install python 3.6 on Shippable.
2017-04-26 15:04:28 +08:00
Matt Clay
e7bb508ad6
Fix var precedence check to support python 3. ( #23552 )
...
* Fix var precedence check to support python 3.
* Run CI sanity tests using python 3.5.
* Disable pylint non-iterator-returned test to pass on python 3.5.
2017-04-13 00:07:40 -07:00
Matt Clay
eec01e2137
Set IO encoding for ansible-test on Shippable.
2017-04-08 23:06:44 -07:00
Matt Clay
9e1bf1c6f2
Update fedora24 Dockerfile to resolve dnf issue.
2017-04-06 16:03:29 -07:00
Matt Clay
dfe8eca3d2
Update Shippable downloader to support new API.
...
- Download run/jobs by URL (old or new).
- Download run metadata.
- Download job metadata.
- Format downloaded JSON.
- Handle and save coverage/testresults download errors.
2017-03-24 14:24:38 -07:00
Matt Clay
12f6beaebd
Remove python2.4 package install from other test.
2017-03-15 16:40:05 -07:00
Matt Clay
869449e288
Add test verification to ansible-test. ( #22636 )
...
* Add unified git diff parser.
* Add metadata and diff handling.
* Add test confidence/verification to bot output.
2017-03-15 12:17:42 -07:00
Matt Clay
89559f78de
Add --failure-ok
option to ansible-test
. ( #22623 )
2017-03-14 16:32:21 -07:00
Matt Clay
a64665ae91
Fix error handling in CI other script.
2017-03-08 01:31:53 -08:00
Matt Clay
3ab344e8bd
Use lint/junit output for compile test. ( #22395 )
...
* Refactor sanity classes for use in all tests.
* Use lint/junit output for compile test.
* Add missing options for compile test.
* Fix early bailout on requirements install.
2017-03-08 00:47:21 -08:00
Matt Clay
9acf551674
Run all other
tests on Shippable before failing.
2017-03-07 20:26:10 -08:00
Matt Clay
8f463fcdd2
Create bot friendly sanity output. ( #22381 )
2017-03-07 14:59:50 -08:00
Matt Clay
72c96b3ac3
Fix encoding issue in shippable downloader.
2017-03-03 16:38:36 -08:00
Matt Clay
d66ce40ecb
Overhaul ansible-test sanity
implementation. ( #22177 )
...
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
2017-03-02 12:36:46 -08:00
Matt Clay
ae2d2860f5
Reduce CI output processing code coverage.
2017-02-27 22:39:35 -08:00
Matt Clay
2804d2e848
Split up unit tests on Shippable. ( #21931 )
...
* Add more verbose output to coverage combine.
* Run unit tests on CI separately by version.
2017-02-24 16:58:56 -08:00
Matt Clay
b2bd75a408
Restore 2008-SP2 and 2008-R2_SP1 CI testing.
2017-02-21 20:55:25 -08:00
Matt Davis
8527013fbe
Complete rewrite of Windows exec wrapper ( #21510 )
...
* supports pipelining for faster execution
* supports become (runas), creates interactive subsession under WinRM batch logon
* supports usage of arbitrary module_utils files
* modular exec wrapper payload supports easier extension
* integrates async wrapper behavior for pipelined/become'd async
* module_utils are loaded as true Powershell modules, no more runtime modifications to module code
2017-02-17 00:09:56 -08:00
Matt Clay
f16ee2841b
Add retries to Shippable CI scripts. ( #21108 )
...
* Add retry.py for auto-retry in CI scripts.
* Add retries to other and units CI scripts.
2017-02-07 16:33:45 -08:00
Matt Clay
f42fa8879b
Temporarily remove Windows Server 2016 from CI.
2017-01-27 11:56:50 -08:00
Matt Clay
7ed67746cf
Enable Windows 2016 on Shippable.
...
Also launch Windows instances on us-east-2 and all others on us-east-1.
2017-01-18 18:37:42 -08:00
Matt Clay
3c51ce56fd
Enable more network support in ansible-test. ( #20372 )
...
* Only start platform instances with tests selected.
* Enable ios on Shippable.
* Show inventory in explain mode.
* Fix indentation of generated network inventory.
* Update classification of network module_utils.
2017-01-17 16:24:05 -08:00
Matt Clay
34bca6fb55
Use distro packaged jinja2 instead of pip version.
2017-01-16 15:52:43 -08:00
Matt Clay
203e375449
Fix python 3.6 install.
2017-01-13 15:30:27 -08:00
Matt Clay
f534573dcf
Enable first network tests on Shippable. ( #20208 )
2017-01-12 18:23:53 -08:00
Matt Clay
cd3fdca540
Switch tests to pytest and ansible-test.
...
- Replace nose usage with pytest.
- Remove legacy Shippable integration.sh.
- Update Makefile to use pytest and ansible-test.
- Convert most yield unit tests to pytest parametrize.
2017-01-11 12:34:59 -08:00
Matt Clay
f340b8d14f
Add test support for python 3.6.
2017-01-10 14:34:42 -08:00
Matt Martz
9048cf2758
Changes to the httptester docker image to support new client cert functionality in tests ( #19912 )
...
* Fix Dockerfile
* Add changes to docker image to support new client cert functionality
* Update repo:tag for docker
2017-01-04 21:07:13 -08:00
Robin Roth
61897fa4b7
Add Dockerfile for opensuse 42.2 ( #19805 )
2017-01-03 11:32:03 -08:00
Matt Clay
68346b128f
Update Fedora tests to use releases 24 and 25.
2017-01-03 11:12:32 -08:00
Matt Clay
165e7c4e61
Add Dockerfile for fedora24 and fedora25.
2017-01-03 10:15:24 -08:00
Matt Clay
422857166b
Remove legacy test alias migration script. ( #19629 )
...
* Require code-smell tests to be files.
* Add sanity check for integration test aliases.
* Remove migration script for test aliases.
2016-12-21 16:46:28 -08:00
Matt Clay
f789a20f29
Update log download url.
2016-12-13 16:26:10 -08:00
Toshio Kuratomi
c6b42028c4
Fixes for uri under python3 and local (non-httptester) testing
2016-12-13 15:46:07 -08:00
Matt Clay
dab2bf7e13
Skip test_uri on python 3.
2016-12-13 13:51:57 -08:00
Lumír 'Frenzy' Balhar
84544ee8fd
Fix AST nodes for Python 3 and enable dependent test_uri ( #18597 )
...
* Enable tests on python 3 for uri
* Added one more node type to SAFE_NODES into safe_eval module.
ast.USub represents unary operators. This is necessary for
parsing some unusual but still valid JSON files during testing
with Python 3.
2016-12-13 07:37:39 -08:00
Matt Clay
f9c4158ff8
Remove core/extras dirs from old PRs during CI.
2016-12-12 12:41:26 -08:00
Matt Clay
580401c02b
Remove legacy test code.
2016-12-08 11:35:20 -05:00
Matt Clay
4da7a7ce0e
Update coverage exclusions. ( #18675 )
...
* Update coverage exclusions.
* Fix logic error in Windows change detection.
2016-11-29 22:46:56 -08:00
Matt Clay
6bbd92e422
Initial ansible-test implementation. ( #18556 )
2016-11-29 21:21:53 -08:00
John R Barker
2b6d5f9f95
Update network-test-runner.sh ( #18672 )
2016-11-29 18:44:31 +00:00
John R Barker
366b7c7691
Network test runner - Phase 1 ( #18524 )
...
* Initial commit of network-test-runner
* remove gist link
* Correctly source hackenv and improve logging
* Remove debug
* Log location
2016-11-29 11:34:37 +00:00
Matt Martz
b79bf14607
Add playbook and packer file for building httptester ( #18107 )
2016-11-17 17:30:30 +00:00