* Add support to Bitwarden Lookup for filtering results by collection id (#5849)
* Debug
* Add support to Bitwarden Lookup for filtering results by collection id (#5849)
* Update comments
* Fix blank line issue
* Fix unit tests for bitwarden lookup plugin. Add changelog fragment file.
* Change collectionId to collection_id parameter on bitwarden plugin
* Fix collection id parameter name when used in bw cli
* Clarify Error message when vault not unlocked
You can be logged into the Bitwarden-CLI, but it can still be locked. This took me several hours to debug, since every time I ran 'bw login' it told me, that I am already logged in.
If you run 'bw unlock' without being logged in, you are prompted to log in.
This clarifies the Error occurring and can drastically reduce debugging time, since you don't have to look into the source code to get an understanding of whats wrong.
* RM: negation
Nobody needs negation
* Update function name
* FIX: tests
* ADD: changelog
* Update changelogs/fragments/5811-clarify-bitwarden-error.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Add Support to Bitwarden Lookup for Custom Fields
This adds support to the Bitwarden lookup for retrieving values from
custom fields, such as api keys.
* Need to Return Whole Record if Field is Not Defined
* whitespace
* Add Changelog Fragment
* Need to Make Sure All Login Fields are Represented
We need to make sure that all login fields are accounted for, since
there will be no other way to retrieve them with this change, and we
don't want to break backwards compatibility. Looking at this code from
the official client,
https://github.com/bitwarden/clients/blob/master/libs/common/spec/models/domain/login.spec.ts,
autofillOnPageLoad might be another login field.
* Update changelogs/fragments/5694-add-custom-fields-to-bitwarden.yml
Clarify changelog fragment
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/lookup/bitwarden.py
Fix logic. Should only error if matches were found, but are missing the custom field.
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Begin building out separate classes to support different op cli versions
Create separet base classes for each major version.
Define the main interface in the base class.
Create methods for getting the current version and instantiating the
appropriate class based on the found version.
* First pass at mostly working CLI version classes
* Correct mismathched parameters
* Update _run() method to allow updating enviroment
This allows passing in the app secret as an env var, which is more
secure than using a command line arg.
* Continuing to improve the interface
* Tear existing tests down to the studs
These tests were based off of the LastPass unit tests. I’m going to
just start from scratch given the new plugin code is vastly diffenent.
* Fix sanity test
* CLI config file path can be None
* Improve required param checking
- only report missing params
- use proper grammer based on number of missing params
* Change assert_logged_in() method return value
Return a boolean value indicating whether or not account is signed in
* Improve full login for v2
Have to do a bit of a dance to avoid hitting the interactive prompt
if there are no accounts configured.
* Remove unused methods
* Add some tests
* Fix linting errors
* Move fixtures to separate file
* Restructure mock test data and add more tests
* Add boilerplate
* Add test scenario for op v2 and increase coverage
* Fix up copyright statements
* Test v1 and v2 in all cases
* Use a more descriptive variable name
* Use docstrings rather than pass in abstract class
This adds coverage to abstract methods with the least amount of hackery.
* Increase test coverage for CLI classes
* Sort test parameters to avoid collection errors
* Update version tested in docs
* Revere test parameter sorting for now
The parameters need to be sorted to avoid the issue in older Python
versions in CI, but I’m having trouble working out how to do that
currently.
* Allow passing kwargs to the lookup module under test
* Favor label over id for v2 when looking for values
Add tests
* Display a warning for section on op v2 or greater
There is no “value” in section fields. If we wanted to support sections
in v2, we would also have to allow specifying the field name in
order to override “value”.
* Move test cases to their own file
Getting a bit unwieldy having it in the test file
* Move output into JSON files fore easier reuse
* Switch to using get_options()
* Add licenses for fixture files
* Use get_option() since get_options() was added in Ansible Core 2.12
* Rearrange fixtures
* Add changelog
* Move common classes to module_utils
* Move common classes back to lookup
The plugin relies on AnsibleLookupError() quite a bit which is not available
in module code.
Remove use of display for errors since section isn’t actually deprecated.
* Properly handle sections
Still room for improvement, but this is at least a start.
* Remove some comments that won’t be addressed
* Make test gathering more deterministic to avoid failures
* Update changelog fragment
* Simple fix for making tests reliable
* Clearer error logging in passwordstore lookup
* Add changelog fragment for passwordstore errmsgs
Co-authored-by: Sylvia van Os <sylvia@hackerchick.me>
* Start using Ansible's config manager to handle options.
* Docs improvements.
* Fix documentation, make options actual lookup options.
* The cyberarkpassword lookup does too strange things.
* The onepassword lookups are converted in #4728, let's not interfere.
* Improve docs.
* Skip shelvefile as well.
* Convert lmdb_kv.
* Convert and fix credstash.
* Convert manifold.
* Drop chef_databag.
* Convert dig.
* Update examples.
* Forgot the most important part.
* Fix lmdb_kv docs.
* Python 2.6 compatibility.
* Convert AnsibleUnicode to str.
* Load lookup with lookup loader.
* Fix environment handling and error message checking.
* Improve docs formatting.
* bitwarden: Add field to search for all item attributes, instead of only name.
* bitwarden: Add change to changelog.
* bitwarden: Update changelog entry.
* Update changelogs/fragments/5297-bitwarden-add-search-field.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/lookup/bitwarden.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/lookup/bitwarden.py
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Ole Pannbacker <opannbacker@cronon.net>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Move licenses to LICENSES/, run add-license.py, add LICENSES/MIT.txt.
* Replace 'Copyright:' with 'Copyright'
sed -i 's|Copyright:\(.*\)|Copyright\1|' $(rg -l 'Copyright:')
Co-authored-by: Maxwell G <gotmax@e.email>
* Fix returnall for gopass
Gopass was always given the --password flag, despite there being no need for this.
* Add changelog fragment
Co-authored-by: Sylvia van Os <sylvia.van.os@politie.nl>
* Fix path detection for gopass
As per fc8c9a2286/docs/features.md (initializing-a-password-store), gopass defaults to ~/.local/share/gopass/stores/root for its password store root location.
However, the user can also override this, and this will be stored in the gopass config file (ed7451678c/docs/config.md (configuration-options)).
This patch ensures that the config setting in gopass is respected, falling back to the default gopass path. pass' behaviour remains unchanged.
* Formatting improvements
Co-authored-by: Felix Fontein <felix@fontein.de>
* Add changelog fragment
* Formatting improvement
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
* Do not ignore tld option in DSV lookup plugin
* add changelog fragment
* Update changelogs/fragments/4911-dsv-honor-tld-option.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* passwordstore: Make compatible with shims, add backend config
This allows using the passwordstore plugin with scripts that wrap other
password managers. Also adds an explicit configuration (`backend` in
`ini` and `passwordstore_backend` in `vars`) to set the backend to `pass`
(the default) or `gopass`, which allows using gopass as the backend
without the need of a wrapper script. Please be aware that gopass
support is currently limited, but will work for basic operations.
Includes integrations tests.
Resolves#4766
* Apply suggestions from code review
* Remove support for the DLV record as the registry was decomissioned
The DLV registry was decomissioned in 2017 (https://www.isc.org/blogs/dlv/) so it's high time we remove support for DLV records.
* Remove DLV deprecation.
Co-authored-by: Felix Fontein <felix@fontein.de>
* Get first found configuration file
There are three valid places to get the configuration.
https://developer.1password.com/docs/cli/about-biometric-unlock#remove-old-account-information
* Use common config class
* Add changelog fragment
* Explicitly use new style classes for Python 2.7 compatibility
This shouldn’t matter for lookups, but does matter for module_utils
and modules since Python 2.7 is still supported on the managed node.
* Update changelogs/fragments/4065-onepassword-config.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Add error handling to check correct SDK version installed
* Fix CI errors
* Added changelog fragment
* Changed exeption type
* Update changelogs fragment
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* passwordstore: Add configurable locking
Passwordstore cannot be accessed safely in parallel, which causes
various issues:
- When accessing the same path, multiple different secrets are
returned when the secret didn't exist (missing=create).
- When accessing the same _or different_ paths, multiple pinentry
dialogs will be spawned by gpg-agent sequentially, having to enter
the password for the same gpg key multiple times in a row.
- Due to issues in gpg dependencies, accessing gpg-agent in parallel
is not reliable, causing plays to fail (this can be fixed by adding
`auto-expand-secmem` to _~/.gnupg/gpg-agent.conf_ though).
These problems have been described in various github issues in the past,
e.g., ansible/ansible#23816 and ansible/ansible#27277.
This cannot be worked around in playbooks by users in a non-error-prone
way.
It is addressed by adding new configuration options:
- lock:
- readwrite: Lock all operations
- write: Only lock write operations (default)
- none: Disable locking
- locktimeout: Time to wait for getting a lock (s/m/h suffix)
(defaults to 15m)
These options can also be set in ansible.cfg, e.g.:
[passwordstore_lookup]
lock=readwrite
locktimeout=30s
Also, add a note about modifying gpg-agent.conf.
* Tidy up locking config
There is no reason why lock configuration should be part of self.paramvals.
Now locking and its configuration happen all in one place.
* Change timeout description wording to the suggested value.
* Rearrange plugin setup, apply PR feedback
The passwordstore lookup plugin depends on parsing GnuPG's
error messages in English language. As a result, detection of
a specific error failes when users set a different locale.
This change corrects this by setting the `LANGUAGE` environment
variable to `C` when invoking `pass`, as this only affects
gettext translations.
See
https://www.gnu.org/software/gettext/manual/html_node/The-LANGUAGE-variable.html
Given a password stored in _path/to/secret_, requesting the password
_path/to_ will literally return `path/to`. This can lead to using
weak passwords by accident/mess up logic in code, based on the
state of the password store.
This is worked around by applying the same logic `pass` uses:
If a password was returned, check if there is a .gpg file it could
have come from. If not, treat it as missing.
Fixesansible-collections/community.general#4185
* Added token parameter for AccessTokenAuthorizer
Parameters username and password are not required anymore because of
this.
* Added changelog fragments
* Apply suggestions from code review
Co-authored-by: Ajpantuso <ajpantuso@gmail.com>
* token authorizer is prioritized
token authorizer is prioritized when token parameter is set
* Apply suggestions from code review
Co-authored-by: Felix Fontein <felix@fontein.de>
* domain optional if token not provided
* Updated examples
- `base_url` is required everywhere
- examples for user, name + domain authorization included
- token authorization included
* Update 3327-tss-token-authorization.yml
* Apply suggestions from code review
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Ajpantuso <ajpantuso@gmail.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Add option for retry_servfail
cf. https://dnspython.readthedocs.io/en/latest/resolver-class.html#dns.resolver.Resolver.retry_servfail
Setting this option to `True` allows for the possibility of the lookup plugin to retry and thereby recover from potentially transient lookup failures, which would otherwise cause the task or play to bail with an unrecoverable exception.
* Create 3247-retry_servfail-for-dig
* documentation for `retry_servfail` option
* Rename 3247-retry_servfail-for-dig to 3247-retry_servfail-for-dig.yaml
* fix whitespace
* Update plugins/lookup/dig.py
Co-authored-by: Ajpantuso <ajpantuso@gmail.com>
* Update plugins/lookup/dig.py
Co-authored-by: Ajpantuso <ajpantuso@gmail.com>
* rm try/except block
Co-authored-by: Ajpantuso <ajpantuso@gmail.com>
* Added fix for bug report in issue #3192
* Added changelog fragment
* Typo fix
* Added Importerror to exception - as req by linters
* Moved the conditional import statement to try/except block
* Updated the plugin to reflect breaking changes introduced in the underlying SDK v1.0.0 update.
* Added Changelog fragment
* Updates based on feedback/review
* Added newline to pass CI
* Added whitepace for linter
* Update changelogs/fragments/3139-tss-lookup-plugin-update-to-make-compatible-with-sdk-v1.yml
Co-authored-by: Ajpantuso <ajpantuso@gmail.com>
Co-authored-by: Ajpantuso <ajpantuso@gmail.com>
* added utf-8 markers to all .py files in plugins/filter
* added utf-8 markers to all .py files in plugins/inventory
* added utf-8 markers to all .py files in plugins/lookup
* Wire token param into consul_api #2124
* Update changelogs/fragments/2124-consul_kv-pass-token.yml
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
* #2124 renamed release fragment to match pr, removed parse_params.
* putting look back in, do some linting #2124
* try more linting
* linting
* try overwriting defaults in parse_params with get_option vals, instead of removing that function completely.
* Revert "back to start, from 2nd approach: allow keyword arguments via parse_params for compatibility."
This reverts commit 748be8e366.
* Revert " linting"
This reverts commit 1d57374c3e.
* Revert " try more linting"
This reverts commit 91c8d06e6a.
* Revert "putting look back in, do some linting #2124"
This reverts commit 87eeec7180.
* Revert " #2124 renamed release fragment to match pr, removed parse_params."
This reverts commit d2869b2f22.
* Revert "Update changelogs/fragments/2124-consul_kv-pass-token.yml"
This reverts commit c50b1cf9d4.
* Revert "Wire token param into consul_api #2124"
This reverts commit b60b6433a8.
* minimal chnages for this PR relative to current upstream.
* superfluous newline in changlog fragment.
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
Add ability to ignore error on missing pass file to allow processing the
output further via another filters (mainly the default filter) without
updating the pass file itself.
It also contains the option to create the pass file, like the option
create=true does.
Finally, it also allows to issue a warning only, if the pass file is not
found.
* Add dependent lookup plugin.
* Use correct YAML booleans.
* Began complete rewrite.
* Only match start of error msg.
* Improve tests.
* Work around old Jinja2 versions.
* Fix metadata.
* Fix filter name.
* convert string returned by plugin to unicode
* add changelog fragment
* fix changelog format
* fix changelog format yet again
Co-authored-by: Anubhav Chakraborty <anubchak@cisco.com>
* with_filetree: use splitext for compatibility with template
The example code given deploys files with their .j2 extensions intact, which is probably not what you want.
* Explain how templates interact with splitext|first
* Update plugins/lookup/filetree.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Don't encourage setting the mode of symlinks
On ext4, maybe most filesystems, symlinks always have the artificial mode of 0777, and `chmod $mode $symlink` *writes through* the symlink to its target file.
An effect of this is that if you deploy a file and a symlink to it (e.g. this common situation: /etc/nginx/sites-available/default and /etc/nginx/sites-enabled/default -> ../sites-available/default) then `with_filetree` will forever first deploy the file with the right mode, then corrupt its mode to 0777, and every redeploy will see a change to fix, forever in a loop.
Probably `file:` should refuse `mode:` on `state: link`s, but in the meantime, avoid recommending it in `filetree`
* Use `follow: false` instead of just the mode.
This should be more cross-compatible.
https://github.com/ansible-collections/community.general/pull/2285#discussion_r616571873
* Update plugins/lookup/filetree.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Adding another example for tss lookup
A more detailed example using self-hosted secrets server as investigated in #1943
* Update plugins/lookup/tss.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Better line breaking
* Update plugins/lookup/tss.py
Seconded!
Co-authored-by: Felix Fontein <felix@fontein.de>
* Remove newline to pass tests
* Update plugins/lookup/tss.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* fix passwordstore.py to be compatible with gopass.
...even when used with create=true.
The same output snippet matches for both, `pass` and `gopass`, but while `pass` returns `1` on a non-existant password, `gopass` returns `10`, or `11`, depending on whether a similar named password was stored.
So I'd propose to change `e.returncode == 1` to `e.returncode != 0` to cover both cases here.
What do you think?
* Update passwordstore.py, fix typo
* Add changelog fragment.
* Update changelogs/fragments/1589-passwordstore-fix-passwordstore.py-to-be-compatible-with-gopass.yaml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update changelogs/fragments/1589-passwordstore-fix-passwordstore.py-to-be-compatible-with-gopass.yaml
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Find the password field out of the fields list
With the command line utility `op` version 1.8, the password field exists, while the fields list is empty. This will look for the desired field without it being listed in the fields list.
* Add changelog fragment
* Update changelogs/fragments/1610-bugfix-onepassword-lookup-plugin.yaml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/lookup/onepassword.py
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update tss.py - multiline for an example
Extended line runs past the side of the browser window
* Moved multiline to after the msg.
Cannot believe I missed that again.
* Updated tss.py
Using > as multiline joiner with spaces
* Remove Google cloud plugins migrated to community.google
* Remove another symlink
* Fix typo for community.general version
* Update changelogs/fragments/1319-google-migration-removal.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update changelogs/fragments/1319-google-migration-removal.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Add fragment for inventory script
* fix yaml formatting
* adjust past relnotes in accordance with removal of google plugins
Co-authored-by: Felix Fontein <felix@fontein.de>
* Added umask option to passwordstore lookup plugin.
* Added umask documentation and changelog fragment.
* Added default values to paramvals within the run method.
* removed blank lines (PEP8)
* Update changelogs/fragments/lookup-passwordstore-umask.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/lookup/passwordstore.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update changelogs/fragments/lookup-passwordstore-umask.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* passwordstore lookup plugin: changelog fragment update
* passing environment variables to subprocess.Popen()
* Update plugins/lookup/passwordstore.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* rm trailing whitespace
* Don't force default umask in the plugin, pass will take care of this.
* remove default from the documentation string
* remove trailing whitespaces
* prevent KeyErrors when checking if key exits in paramvals.
* Update plugins/lookup/passwordstore.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Fix for TypeError
* revert back to old directory test
Co-authored-by: bratw0rst <c.chmiel@speakup.nl>
Co-authored-by: Felix Fontein <felix@fontein.de>
As per the plugin documentation and the Hashicorp Vault documentation (https://www.vaultproject.io/docs/auth/approle#secretid)
secret_id is not mandatory.
Moreover, using this lookup plugin without a secret_id used to work in
Ansible 2.9.
Co-authored-by: Jonathan Piron <jonathanpiron@gmail.com>
* Add support for Hashicorp Vault JWT auth
* Add support for HashiCorp Vault JWT auth (continued)
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Co-authored-by: Mike Brancato <mike@mikebrancato.com>
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* callback_type -> type.
* Mark authors as unknown.
* Add author field forgotten in #627.
* Fix author entries.
* Add author field forgotten in #127.
* Fix some types.
As per https://learn.hashicorp.com/tutorials/vault/namespaces, setting VAULT_NAMESPACE env var is a completely supported mechanism to make all vault command use said namespace, so hashi_vault lookup function should do the same.
Co-authored-by: Holt Wilkins <hwilkins@palantir.com>
* Fix deprecation of callables.
* Fix various sanity errors.
* Revert callback_type -> type transform.
* Fix stat_result times: these are float according to https://github.com/python/typeshed/blob/master/stdlib/3/os/__init__.pyi
* Apply suggestions from code review
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
