* switch to boto3 and add support for application ELBs with target groups.
* use py23 compatible dict iterator.
* removing commented out fail_json calls
utilize sets to simplify logic
remove setting a redundant variable
add bounds checking in two places
add AWSRetry decorator - do we want this for other functions too?
change xrange to range so python3 doesn't fail
remove sorting lists of dicts; in python2 this returns None, in python3 this fails
* remove error variable from traceback.format_exc
* Remove boto2-style calls brought in by rebase
Old boto-style calls to `as_group` attributes break in boto3
Also remove module from legacy-PEP8 list
* Add parameter to target_group_arn option
* Fix HAS_BOTO3 check
* use tags.items() instead of iteritems
* import botocore
* Fixed bugs in deleting autoscaling groups
* make changes in deleting autoscaling groups pep8
* more pep8
* fix version
* fix bugs so local integration tests run
* fix launch config check
* reflect changed status for ASG updates
* Fix existing exception handling and use traceback.
Fix imports
* line length
* Fix notification setup
* Fix mutually exclusive arguments
Only one of the AvailabilityZones and VPCZoneIdentifier arguments should be provided to the CreateAutoScalingGroup call.
* Allow desired_capacity, min_size, max_size, launch_config_name to be derived from the existing ASG if not specified
Remove code updating dict after ASG already uses it
* Add new helper function for comparing AWS tag key pair dicts. Also modify boto3_tag_list_to_ansible_dict function to be more generic when looking for key names because AWS sometimes uses 'Key', sometimes 'TagKey' and who knows what the future holds! Fixed modules to work with changes.
* Review changes
* Add some more doc to GUIDELINES for tags and fix var name for snaked values in ec2_group_facts
* allow elb scheme to be updated by restarting the resource
use ensure_gone and require wait option
* fix pep8
* fix yaml
* requested changes
* fix fail_json
- Use `fail_json` for `get_all_load_balancers` unhandled exception.
- Do not mask error message on failure of `enable_zones`.
- Do not mask error message on failure of `disable_zones`.
Also added/removed blank lines to improve PEP 8 compliance.
* fix documentation and correct exception handling
* follow AWS exception guidelines
* fix parameter_group_family req; only needed when creating cache parameter group
make pep8 and remove from legacy files
update module to support more standard state=present/absent syntax
update module to use required_if, required_together, mutually_exclusive functions where possible
per ryansb review: make documentation section more clear, fix some extra quotes, remove FIXME comment
pre willthames review: force private_zone to True if vpc_id is set and fix word wrap
* Support check mode in ec2_vpc_dhcp_options_facts
As a facts module, ec2_vpc_dhcp_options_facts supports check mode
by default
* ec2_vpc_dhcp_options_facts tidy up
Use named method imports, move imports to top of code
Use shared code to handle filters and tags
Use snake case for parameter names while retaining backward compatibility
This fixes issue when list from module contains more than one element.
Ansible and/or boto may put same elements in list in different order,
thus resulting task as changed.
Fixes#3310
* allow split horizon for route53_zone and refactor
* fix documentation
remove comment
fix version_added
* Remove unused imports
* Only include zone as matching if it has the same privacy setting
* Use `.endswith` instead of indexing into a string
* Update public zone behavior to only create new if there is no matching public zone
* Remove from legacy PEP8 files
* Add ability to use either sec group name or sec group id or combination to ec2_lc. Also fix header size in AWS Guidelines
* Fix ec2 import to only import neccessary packages. Remove pep8 double line
* re-raise exception as expected
The current ThrottlingException handling code hides other actual exceptions from the user, and basically goes infinite loop instead. eg when the api caller doesn't have permission (example below) to use the api, ansible effectively hangs. adding the re-raise stops execution and shows the error to the command line as expected
can test by removing permission to the efs api, and calling the efs: module
The error was: botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the DescribeFileSystems operation: User: <x> is not authorized to perform: elasticfilesystem:DescribeFileSystems on the specified resource
* PEP8 fix
* Remove DryRun parameter in ec2_vpc_igw_facts check_mode
Using DryRun in check mode causes errors, and is not required
(as nothing changes when calling describe_internet_gateways)
Prevents the following error:
```
{"changed": false,
"failed": true,
"msg": "An error occurred (DryRunOperation) when calling
the DescribeInternetGateways operation: Request
would have succeeded, but DryRun flag is set."}
```
* ec2_vpc_igw_facts pep8 tidy up
* [cloud][tests] Create fixtures for using placebo to test boto3-using modules
* Use pytest's importorskip instead of manually skipping on missing deps
* Fix imports in cloudformation module
* Delete unused code
* Add maybe_sleep fixtures to speed up recorded test runs
* Build basic placebo-CFN tests
* Commit placebo recordings of basic stack operations
* Add placebo to test-requires
* Allow unit tests to run regardless of environment by setting a default region
* Use explicit relative import for Python 3 compat
* Use __name__ attribute that works on Python 2 and 3
* add else statement to ensure instids is set
set res_list to None to avoid UnboundLocalError and fix iteration over a nonetype by adding an empty tuple
* make res_list empty tuple by default and check for instids before setting tags (fails otherwise)
Set encrypted to default False, rather than None
Otherwise you get:
```
Invalid type for parameter Encrypted, value: None, type: <type 'NoneType'>, valid types: <type 'bool'>)
```
* update integration tests for updated boto exception message
* integration tests fail on both "test credential" test cases
exception bubbles out of module. instead catch and wrap
* ec2_group does not support updating a security group's description
AWS security group descriptions are immutable.
if ec2_group finds a group that matches by name, but the descriptions do not match, the module does not support this case
previously it would check if the group was used, but would not do anything if it was
old behavior was erroneous because it could make a user expect that the description change of a group was fine when in fact it did not occur
also, it made an expensive check against all ec2 instances for no good reason
* comments not doc strings
* else must have pass w/o doc-string statement
* Catch specific BotoServerException, give context around error when fetching SGs
* python3 compatible exception blocks
* add traceback to fail_json
* two blank lines before first function
purge_subnets|routes were introduced recently.
ensure_subnet_associations now takes purge_subnets as a parameter. A
call to this function was missed when introducing this feature. With
out, results in a "got 5 expected 6" error.
Allows to health-check two services on one host. Before, asking for 2 health checks being present for `http://myhost:8888` and `http://myhost:9999` resulted in just a single health check, as this module considered the two specs as equal.
Bonus: route53_health_check now passes style guidelines
* Fix get_s3_connection (fixes#22317)
Override aws_connect_kwargs rather than prepending to them. Should fix an issue in which `calling_format` is set twice in the kwargs passed to `boto.connect_s3` or `S3Connection` if a bucket name contains a `.`
* Revert "Fix get_s3_connection (fixes#22317)"
This reverts commit 7f61b8bebd2929940495204f1a98d660a55985d8.
* implements alternative way of fixing issue with aws_connect_kwargs for rgw and fakes3 (fixes 22317)
* add comment to explain why the keys are being removed from aws_connect_kwargs
* remove trailing whitespace on comment line
* don't modify an eni without eni_id
fix pep8
remove ec2_eni from pep8 legacy files
remove a couple unused variables in main()
fix the documentation
* fix yaml
* making ec2 pep8
* remove ec2 from pep8 legacy files
* missed a couple
* fix imports and remove iteritems
* making group_id and group_name mutually exclusive and fixing whitespace
Changes to the metadata format were approved here:
https://github.com/ansible/proposals/issues/54
* Update documentation to the new metadata format
* Changes to metadata-tool to account for new metadata
* Add GPL license header
* Add upgrade subcommand to upgrade metadata version
* Change default metadata to the new format
* Fix exclusion of non-modules from the metadata report
* Fix ansible-doc for new module metadata
* Exclude metadata version from ansible-doc output
* Fix website docs generation for the new metadata
* Update metadata schema in valiate-modules test
* Update the metadata in all modules to the new version
Currently the documentation utilizes with_subelements, but it does not parse the results correctly. By changing to with_items: and specifying the proper list, we are able to tag the instances as expected.
Unclear how useful it is to still be catching BotoServerError exceptions
generally, especially when we are now hopefully handling most instances
of throttling, but kept it as it was already in the code. Moved to main
as catching BotoServerErrors inside individual functions will bypass any
AWSRetry.backoff decorator.
Move additional imports to top.
Use python2.6 formatting
A bit of cleanup on imports
Make one more string interpolation Python 2.6 compatible.
* Fix connection_draining idempotency in ec2_elb_lb
Ensure connection_draining types are equivalent when comparing
whether or not connection_draining is being changed.
This means that running `ec2_elb_lb` with connection_draining
set a second time will now report `changed=False`
* ec2_elb_lb: fixed latest Ansible standards compliance
Update to current pep8 standards, fix module imports
and remove module from exclusion file
* fix module doc fields
* More module docs corrections
* More module docs corrections
* More module docs corrections
* More module docs corrections
* correct aliases
* Review comments
* Must quote ':'
* More authors
* Use suboptions:
* restore type: bool
* type should be in the same place
* More tidyups
* authors
* Use suboptions
* revert
* remove duplicate author
* More issues post rebase
* description is only required when group state is present
also note that AWS requires a non-empty description when creating a security group
* clarify description requirement
1. Check less aggressively for resources that take a long time to return
(some resources can take an hour or more).
2. Handle throttling when checking for resources, and back off quickly
when it occurs.
* Add user_data_path parameter to ec2_lc module
* Improve user_data_path parameter documentation
- Specify mutual exclusivity
- Specify `version_added`
- Change module parameter type to `path`
* Use correct comparison for user_data result
* Include traceback on error and use with block
* Only hide user data if provided in file
ensure_tags fails if `tags` is None rather than an empty
dict. Ensure that not passing `tags` parameter is equivalent
to passing an empty dict.
Fixes#21778
Fixes ansible#21796 Prevent users from deleting buckets rather than objects by making object parameter and mode=delobj mutually exclusive in task.
https://github.com/ansible/ansible/issues/21796
The list_elbs call to boto doesn't use any pagination, so any time there
are more ELBs than the API page size, this module will fail. This change
uses the `next_token` attribute of `ResultSet` to check if there are
still more ELBs to return.
Fixes#21361
* Make more use of AWS Exception information
* Use pythonic dict assignments
* Improve documentation formatting to make option names clearer
in descriptions
* Meet ansible line length requirements
* Tidy up flake8 messages
Set the `force` flag to false to:
- protect existing Keys from being overwritten
- check if a Key was already existing by evaluating
the returned `changed` status
CentOS's cloud image wiki page (wiki.centos.org/Cloud/AWS) gives an
example of using their product code to search for AMIs. This commit
adds an optional parameter to ec2_ami_find to allow this filtering.
If versioning is not enabled, then `get_bucket_versioning`
can return an empty dict.
If that happens, the code to enable versioning should still
run!
The logic for suspending versioning was also incorrect, so
have updated that too.
Fixes#20491
Introduces `ignore_nonexistent_bucket` option that does not require ListBuckets permission, so less permissioned IAM roles can still be used with this module.
Fixes#20501
Don't try to create tags on a vpc that you've just removed.
Avoids
```
"msg": "An error occurred (InvalidParameterValue) when calling the CreateTags operation: You must specify one or more tags to create"
```
Although not quite sure why the `create_tags` was being called
as `module.params.get('tags')` *should* have returned `None`.
* Adding an elasticache snapshot module.
Allows user to create, copy, or delete a snapshot.
* Removing unnecessary function
* Make indentation uniform.
* Making requested changes.
Fixing PEP8
Adding a more graceful fail for delete() if the snapshot's state valid (such as when it is in the process of being created).
* PEP8
* Fixing some formatting
move imports
fix parameter alignment
* move imports to the top of the file below documentation
Replaces the usage of expanduser with the type 'path' for the
dest and object options as that functionality is available
in that type automatically.
patch related to #12263
* initial commit of cloudfront_facts.py
* modification as per review from @georgepsarakis
* fixed shippable build error
* fixed shippable build error
* removed wildcard imports and replaced with specific imports from ansible.module_utils.ec2 as advised by @georgepsarakis
* renamed all instances of cloud_front_origin_access_identity to origin_access_identity as advised by @georgepsarakis
* show facts based on alias and distribution id for easy referencing as advised by @ryansb. have done for both distribution and distribution_config
* fixed incorrect logic for default list_distributions, fixed list_distributions_by_web_acl - wasn't passing web_acl_id, fixed list_invalidations keyword args missing DistributionId
* fixed last fact added clobbering all previous facts for an alias or an id
* removed list_ prefix from list keys
* removed unnecessary boto fields. made list_distributions and list_streaming_distributions dictionaries with id/alias as key. fixed list_invalidations.
* initial commit of cloudfront_facts summary
* checks for empty list returned from boto, standardised list naming
* neatened up parameters
* added summary documentation
* refactoring of functions for modularity
* refactoring, neatening code, fix for if cname not present, added try-catch blocks
* more refactoring, cleaning
* more cleaning, allowed streaming distributions to be found by domain name alias
* removed unnecessary line
* fixed tabs
* fixed indentation
* removed trailing whitespace
* removed more whitespace
* more refactoring, modified where invalidations dict is set
* added ETag to summary
* refactored summary list
* moved list out of for loop
* trailing white space
* reverted line lengthening as advised by @willthames
* modified exceptions to multi line as advised by @willthames
* reverted variable spacing to be more pythonic'
* reverted spacing in YAML as advised by @ryansb
* reverted line spacing for parameters for correct blame attribution as advised by @ryansb
* removed white space
* more white space
* reverted line spacings for parameters as advised by @ryansb
* removed spaces between parameters as advised by @ryansb
* Avoid having module documentation links to itself
A lot of modules use M(own_module) in their documentation causing a link
in the documentation to itself.
* Make note more clear now
* Removing cmp to be compatible with Python3
* fix syntax
* Added an overlooked return
* Fixing making things overly complicated
* Simplifying since tags will always be hashable. Don't need to use cmp at all.
* Adding environment variable support for lambda.py in response to feature idea #20479.
Plus a 1-character bug fix.
* fix yaml syntax
* Fixing option name, adding alias, and fixing a line to allow the user to delete environment variables by setting an empty dict.
* Update validate-modules
* Validates ANSIBLE_METADATA
* Ensures imports happen after documentation vars
* Some pep8 cleanup
* Clean up some left over unneeded code
* Update modules for new module guidelines and validate-modules checks
* Update imports for ec2_vpc_route_table and ec2_vpc_nat_gateway
Boolean options that default as `None` but are set to `False` by the user were ignored on update. This change checks to distinguish None & False so that options like multi_az can be turned off during an update.
* Modifying how optional parameters are handled in rds.py. Fixes#20370
Allowing options to be set to false/no. Previously ignored unless set to true/yes.
Added a conditional for invalid parameters since the default is false instead of null for some options (e.g. force_failover, apply_immediately, upgrade).
* Making requested revision.
* Disassociate subnets from route tables before deletion
If a route table still has subnets associated with it, it will fail
to delete:
```
"msg": "The routeTable 'rtb-abcd1234' has dependencies and cannot be deleted."
```
Avoid this by disassociating subnets before route table deletion
* Fix ec2_vpc_route_table flake8 complaints
* Check if EIP exists before deleting it
After deleting the NAT gateway, the EIP sometimes seems to
cease to exist afterwards. Check if it exists before deleting it.
Otherwise you get:
```
Failed to release EIP eipalloc-abdc1234: An error occurred (InvalidAllocationID.NotFound) \
when calling the ReleaseAddress operation: The allocation ID 'eipalloc-abcd1234' does not \
exist", "success": false}
```
* Fix flake8 errors with ec2_vpc_nat_gateway
* added ec2_vpc_igw_facts module
* added cr at end of file
* corrected import json in wrong location
* corrected version added
* added snake_case conversion
* updated documentation and fixed for python 3'
* ec2_vpc_igw_facts: simplify logic
Make module arguments more 'Ansiblish'
Remove unnecessary intermediate variables in results generation
Use `ansible_dict_to_boto3_filter_list` rather than duplicating logic
Use `check_mode` rather than pass a `dryrun` argument
Update for flake8 improvements
* updated documentation
Update exception handling, remove use of iteritems
Update for better flake8 compliance
Use ansible_dict_to_boto3_filter_list rather than
duplicating its implementation
* new AWS module for ec2 VPC vgw facts
* fixed documentation as suggested by reviewers
* changed iteritems to items for py 3
* improvements to bring in line with #19787
* corrected documentation
* updated doco and exception path
* Deprecate ec2_vpc module
The deprecation of ec2_vpc module has been discussed for
2 years and is causing duplication of effort as changes
are implemented for ec2_vpc rather than for the newer
alternatives
* Improve module deprecation documentation
Update the developing modules documentation with the latest
instructions on how to deprecate a module.
* Fixes: #3217 - Add a new parameter for the kms_key_id to the ec2_vol module.
* Fixes: #3217 - Correcting comment string as requested.
* Fixes: #3217 - Adding boto version when kms_key_id is used. Also re-adding accidentally removed comment line.
* Cleanup of EBS volume key pull request
* Adding support for Amazon ECR
This patch adds a new module named ecr, which can create, update or
destroy Amazon EC2 Container Registries. It also handles the management
of ECR policies.
* ecs_ecr: addressed review feeback
* Renaming ecr to ecs_ecr
* Fixed docs
* Removed bad doc about empty string handling
* Added example of `delete_policy`
* Removed `policy_text` option; switched policy to `json` type so
it can accept string or dict
* Added support for specifying registry_id
* Added explicit else after returned if clauses
* Added `force_set_policy` option
* Improved `set_repository_policy` error handling
* Fixed policy comparisons when AWS doesn't keep the ordering stable
* Moved `boto_exception` into the module
* initial commit of cloudfront_facts.py
* modification as per review from @georgepsarakis
* fixed shippable build error
* fixed shippable build error
* removed wildcard imports and replaced with specific imports from ansible.module_utils.ec2 as advised by @georgepsarakis
* renamed all instances of cloud_front_origin_access_identity to origin_access_identity as advised by @georgepsarakis
* added input option requirements where missing. fixed a typo in an error message, added '.' to the end of sentences.
* fixed typo in documentation
* simplified paginated_response method as advised by @georgepsarakis
* set default option to list_defaults when no option specified as advised by @ryansb
* bumped version to 2.3 as advised by @ryansb
* removed double-nesting of ansible_facts as advised by @ryansb
* show facts based on alias and distribution id for easy referencing as advised by @ryansb. have done for both distribution and distribution_config
* made comments clearer
* fixed incorrect logic for default list_distributions, fixed list_distributions_by_web_acl - wasn't passing web_acl_id, fixed list_invalidations keyword args missing DistributionId
* neatened up logic for list_distributions default
* facts now return Etag as well as specific facts. modified paginated_response to handle this. also all requests that use distribution id also list the facts under the corresponding alias now.
* fixed last fact added clobbering all previous facts for an alias or an id
* removed list_ prefix from list keys
* removed unnecessary boto fields. made list_distributions and list_streaming_distributions dictionaries with id/alias as key. fixed list_invalidations.
* fixed incorrectly named list_distributions_by_web_acl to ..web_acl_id. added id/alias dict for list_distributions_by_web_acl_id
* minor doc changes
* removed merge tag
* fixed more of merge
The main purpose of this PR is to add the subnet associations to the
dict returned by ec2_vpc_route_table_facts. This commit also
re-formats code to make it PEP8 compliant.
New module by @tedder for handling granting/revoking access to KMS secrets.
For example:
```
- name: grant user-style access to production secrets
kms:
args:
mode: grant
key_alias: "alias/my_production_secrets"
role_name: "prod-appServerRole-1R5AQG2BSEL6L"
grant_types: "role,role grant"
```
- Consistent capitalisation in the descriptions
- Removed redundant 'optional' notes when this is covered by the Boolean `optional` column
- Clarified `instance_id` description
* set is_public 'true' or 'false'
Despite being a boolean property, https://github.com/melta/boto/blob/master/boto/ec2/image.py:63 sets is_public = True only if the argument is passed in as the string 'true'. Likewise for False/'false'.
This is a workaround for that bug in boto2, to allow the documented parameter to work with valid yaml values.
fixes#5600
* only set is_public if true
Actual doc give this error
```
fatal: [localhost]: FAILED! => {
"changed": false,
"failed": true,
"invocation": {
"module_args": {
"command": "facts",
"register": "new_database_facts"
},
"module_name": "rds"
},
"msg": "unsupported parameter for module: register"
}
```
Register should be at the module `rds` level and not at args level in this example :
```
- rds:
command: facts
instance_name: new-database
- register: new_database_facts
+ register: new_database_facts
```
* Add new parameters to taskdefinition module - network_mode and task_role_arn
* Add version_added field for doco
* Change version_added parameter to 2.3
