ansible/community.general
1
0
Fork 0
mirror of https://github.com/ansible-collections/community.general.git synced 2024-09-14 20:13:21 +02:00
Code Releases Activity

Galaxy 2.0

Browse source
This commit is contained in:
chouseknecht 2015-12-09 10:51:12 -05:00
parent 0719eb3e2d
commit 4f84769a17
11 changed files with 949 additions and 112 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

291
docsite/rst/galaxy.rst
View file

@ -8,7 +8,7 @@ Ansible Galaxy
The Website
```````````
The website `Ansible Galaxy <https://galaxy.ansible.com>`_, is a free site for finding, downloading, rating, and reviewing all kinds of community developed Ansible roles and can be a great way to get a jumpstart on your automation projects.
The website `Ansible Galaxy <https://galaxy.ansible.com>`_, is a free site for finding, downloading, and sharing community developed Ansible roles. Downloading roles from Galaxy is a great way to jumpstart your automation projects.
You can sign up with social auth and use the download client 'ansible-galaxy' which is included in Ansible 1.4.2 and later.
@ -24,7 +24,7 @@ Installing Roles
The most obvious is downloading roles from the Ansible Galaxy website::
ansible-galaxy install username.rolename
$ ansible-galaxy install username.rolename
.. _galaxy_cli_roles_path:
@ -33,23 +33,16 @@ roles_path
You can specify a particular directory where you want the downloaded roles to be placed::
ansible-galaxy install username.role -p ~/Code/ansible_roles/
$ ansible-galaxy install username.role -p ~/Code/ansible_roles/
This can be useful if you have a master folder that contains ansible galaxy roles shared across several projects. The default is the roles_path configured in your ansible.cfg file (/etc/ansible/roles if not configured).
Building out Role Scaffolding
-----------------------------
It can also be used to initialize the base structure of a new role, saving time on creating the various directories and main.yml files a role requires::
ansible-galaxy init rolename
Installing Multiple Roles From A File
-------------------------------------
=====================================
To install multiple roles, the ansible-galaxy CLI can be fed a requirements file. All versions of ansible allow the following syntax for installing roles from the Ansible Galaxy website::
ansible-galaxy install -r requirements.txt
$ ansible-galaxy install -r requirements.txt
Where the requirements.txt looks like::
@ -64,7 +57,7 @@ To request specific versions (tags) of a role, use this syntax in the roles file
Available versions will be listed on the Ansible Galaxy webpage for that role.
Advanced Control over Role Requirements Files
---------------------------------------------
=============================================
For more advanced control over where to download roles from, including support for remote repositories, Ansible 1.8 and later support a new YAML format for the role requirements file, which must end in a 'yml' extension. It works like this::
@ -121,3 +114,275 @@ Roles pulled from galaxy work as with other SCM sourced roles above. To download
`irc.freenode.net <http://irc.freenode.net>`_
#ansible IRC chat channel
Building Role Scaffolding
-------------------------
Use the init command to initialize the base structure of a new role, saving time on creating the various directories and main.yml files a role requires::
$ ansible-galaxy init rolename
The above will create the following directory structure in the current working directory:
::
README.md
.travsis.yml
defaults/
main.yml
files/
handlers/
main.yml
meta/
main.yml
templates/
tests/
inventory
test.yml
vars/
main.yml
.. note::
.travis.yml and tests/ are new in Ansible 2.0
If a directory matching the name of the role already exists in the current working directory, the init command will result in an error. To ignore the error use the --force option. Force will create the above subdirectories and files, replacing anything that matches.
Search for Roles
----------------
The search command provides for querying the Galaxy database, allowing for searching by tags, platforms, author and multiple keywords. For example:
::
$ ansible-galaxy search elasticsearch --author geerlingguy
The search command will return a list of the first 1000 results matching your search:
::
Found 2 roles matching your search:
Name Description
---- -----------
geerlingguy.elasticsearch Elasticsearch for Linux.
geerlingguy.elasticsearch-curator Elasticsearch curator for Linux.
.. note::
The format of results pictured here is new in Ansible 2.0.
Get More Information About a Role
---------------------------------
Use the info command To view more detail about a specific role:
::
$ ansible-galaxy info username.role_name
This returns everything found in Galaxy for the role:
::
Role: username.rolename
description: Installs and configures a thing, a distributed, highly available NoSQL thing.
active: True
commit: c01947b7bc89ebc0b8a2e298b87ab416aed9dd57
commit_message: Adding travis
commit_url: https://github.com/username/repo_name/commit/c01947b7bc89ebc0b8a2e298b87ab
company: My Company, Inc.
created: 2015-12-08T14:17:52.773Z
download_count: 1
forks_count: 0
github_branch:
github_repo: repo_name
github_user: username
id: 6381
is_valid: True
issue_tracker_url:
license: Apache
min_ansible_version: 1.4
modified: 2015-12-08T18:43:49.085Z
namespace: username
open_issues_count: 0
path: /Users/username/projects/roles
scm: None
src: username.repo_name
stargazers_count: 0
travis_status_url: https://travis-ci.org/username/repo_name.svg?branch=master
version:
watchers_count: 1
.. note::
The format of results pictured here is new in Ansible 2.0.
List Installed Roles
--------------------
The list command shows the name and version of each role installed in roles_path.
::
$ ansible-galaxy list
- chouseknecht.role-install_mongod, master
- chouseknecht.test-role-1, v1.0.2
- chrismeyersfsu.role-iptables, master
- chrismeyersfsu.role-required_vars, master
Remove an Installed Role
------------------------
The remove command will delete a role from roles_path:
::
$ ansible-galaxy remove username.rolename
Authenticate with Galaxy
------------------------
To use the import, delete and setup commands authentication with Galaxy is required. The login command will authenticate the user,retrieve a token from Galaxy, and store it in the user's home directory.
::
$ ansible-galaxy login
We need your Github login to identify you.
This information will not be sent to Galaxy, only to api.github.com.
The password will not be displayed.
Use --github-token if you do not want to enter your password.
Github Username: dsmith
Password for dsmith:
Succesfully logged into Galaxy as dsmith
As depicted above, the login command prompts for a GitHub username and password. It does NOT send your password to Galaxy. It actually authenticates with GitHub and creates a personal access token. It then sends the personal access token to Galaxy, which in turn verifies that you are you and returns a Galaxy access token. After authentication completes the GitHub personal access token is destroyed.
If you do not wish to use your GitHub password, or if you have two-factor authentication enabled with GitHub, use the --github-token option to pass a personal access token that you create. Log into GitHub, go to Settings and click on Personal Access Token to create a token.
Import a Role
-------------
Roles can be imported using ansible-galaxy. The import command expects that the user previously authenticated with Galaxy using the login command.
Import any GitHub repo you have access to:
::
$ ansible-galaxy import github_user github_repo
By default the command will wait for the role to be imported by Galaxy, displaying the results as the import progresses:
::
Successfully submitted import request 41
Starting import 41: role_name=myrole repo=githubuser/ansible-role-repo ref=
Retrieving Github repo githubuser/ansible-role-repo
Accessing branch: master
Parsing and validating meta/main.yml
Parsing galaxy_tags
Parsing platforms
Adding dependencies
Parsing and validating README.md
Adding repo tags as role versions
Import completed
Status SUCCESS : warnings=0 errors=0
Use the --branch option to import a specific branch. If not specified, the default branch for the repo will be used.
If the --no-wait option is present, the command will not wait for results. Results of the most recent import for any of your roles is available on the Galaxy web site under My Imports.
.. note::
The import command is only available in Ansible 2.0.
Delete a Role
-------------
Remove a role from the Galaxy web site using the delete command. You can delete any role that you have access to in GitHub. The delete command expects that the user previously authenticated with Galaxy using the login command.
::
ansible-galaxy delete github_user github_repo
This only removes the role from Galaxy. It does not impact the actual GitHub repo.
.. note::
The delete command is only available in Ansible 2.0.
Setup Travis Integerations
--------------------------
Using the setup command you can enable notifications from `travis <http://travis-ci.org>`_. The setup command expects that the user previously authenticated with Galaxy using the login command.
::
$ ansible-galaxy setup travis github_user github_repo xxxtravistokenxxx
Added integration for travis chouseknecht/ansible-role-sendmail
The setup command requires your Travis token. The Travis token is not stored in Galaxy. It is used along with the GitHub username and repo to create a hash as described in `the Travis documentation <https://docs.travis-ci.com/user/notifications/>`_. The calculated hash is stored in Galaxy and used to verify notifications received from Travis.
The setup command enables Galaxy to respond to notifications. Follow the `Travis getting started guide <https://docs.travis-ci.com/user/getting-started/>`_ to enable the Travis build process for the role repository.
When you create your .travis.yml file add the following to cause Travis to notify Galaxy when a build completes:
::
notifications:
webhooks: https://galaxy.ansible.com/api/v1/notifications/
.. note::
The setup command is only available in Ansible 2.0.
List Travis Integrtions
=======================
Use the --list option to display your Travis integrations:
::
$ ansible-galaxy setup --list
ID Source Repo
---------- ---------- ----------
2 travis github_user/github_repo
1 travis github_user/github_repo
Remove Travis Integrations
==========================
Use the --remove option to disable a Travis integration:
::
$ ansible-galaxy setup --remove ID
Provide the ID of the integration you want disabled. Use the --list option to get the ID.

322
lib/ansible/cli/galaxy.py
View file

@ -22,10 +22,11 @@
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import os
import os.path
import sys
import yaml
import json
import time
from collections import defaultdict
from jinja2 import Environment
@ -36,7 +37,10 @@ from ansible.errors import AnsibleError, AnsibleOptionsError
from ansible.galaxy import Galaxy
from ansible.galaxy.api import GalaxyAPI
from ansible.galaxy.role import GalaxyRole
from ansible.galaxy.login import GalaxyLogin
from ansible.galaxy.token import GalaxyToken
from ansible.playbook.role.requirement import RoleRequirement
from ansible.module_utils.urls import open_url
try:
from __main__ import display
@ -44,18 +48,52 @@ except ImportError:
from ansible.utils.display import Display
display = Display()
class GalaxyCLI(CLI):
VALID_ACTIONS = ("init", "info", "install", "list", "remove", "search")
available_commands = {
"delete": "remove a role from Galaxy",
"import": "add a role contained in a GitHub repo to Galaxy",
"info": "display details about a particular role",
"init": "create a role directory structure in your roles path",
"install": "download a role into your roles path",
"list": "enumerate roles found in your roles path",
"login": "authenticate with Galaxy API and store the token",
"remove": "delete a role from your roles path",
"search": "query the Galaxy API",
"setup": "add a TravisCI integration to Galaxy",
}
SKIP_INFO_KEYS = ("name", "description", "readme_html", "related", "summary_fields", "average_aw_composite", "average_aw_score", "url" )
def __init__(self, args):
self.VALID_ACTIONS = self.available_commands.keys()
self.VALID_ACTIONS.sort()
self.api = None
self.galaxy = None
super(GalaxyCLI, self).__init__(args)
def set_action(self):
"""
Get the action the user wants to execute from the sys argv list.
"""
for i in range(0,len(self.args)):
arg = self.args[i]
if arg in self.VALID_ACTIONS:
self.action = arg
del self.args[i]
break
if not self.action:
self.show_available_actions()
def show_available_actions(self):
# list available commands
display.display(u'\n' + "usage: ansible-galaxy COMMAND [--help] [options] ...")
display.display(u'\n' + "availabe commands:" + u'\n\n')
for key in self.VALID_ACTIONS:
display.display(u'\t' + "%-12s %s" % (key, self.available_commands[key]))
display.display(' ')
def parse(self):
''' create an options parser for bin/ansible '''
@ -63,11 +101,21 @@ class GalaxyCLI(CLI):
usage = "usage: %%prog [%s] [--help] [options] ..." % "|".join(self.VALID_ACTIONS),
epilog = "\nSee '%s <command> --help' for more information on a specific command.\n\n" % os.path.basename(sys.argv[0])
)
self.set_action()
# options specific to actions
if self.action == "info":
if self.action == "delete":
self.parser.set_usage("usage: %prog delete [options] github_user github_repo")
elif self.action == "import":
self.parser.set_usage("usage: %prog import [options] github_user github_repo")
self.parser.add_option('-n', '--no-wait', dest='wait', action='store_false', default=True,
help='Don\'t wait for import results.')
self.parser.add_option('-b', '--branch', dest='reference',
help='The name of a branch to import. Defaults to the repository\'s default branch (usually master)')
self.parser.add_option('-t', '--status', dest='check_status', action='store_true', default=False,
help='Check the status of the most recent import request for given github_user/github_repo.')
elif self.action == "info":
self.parser.set_usage("usage: %prog info [options] role_name[,version]")
elif self.action == "init":
self.parser.set_usage("usage: %prog init [options] role_name")
@ -83,27 +131,40 @@ class GalaxyCLI(CLI):
self.parser.add_option('-n', '--no-deps', dest='no_deps', action='store_true', default=False,
help='Don\'t download roles listed as dependencies')
self.parser.add_option('-r', '--role-file', dest='role_file',
help='A file containing a list of roles to be imported')
help='A file containing a list of roles to be imported')
elif self.action == "remove":
self.parser.set_usage("usage: %prog remove role1 role2 ...")
elif self.action == "list":
self.parser.set_usage("usage: %prog list [role_name]")
elif self.action == "login":
self.parser.set_usage("usage: %prog login [options]")
self.parser.add_option('-g','--github-token', dest='token', default=None,
help='Identify with github token rather than username and password.')
elif self.action == "search":
self.parser.add_option('--platforms', dest='platforms',
help='list of OS platforms to filter by')
self.parser.add_option('--galaxy-tags', dest='tags',
help='list of galaxy tags to filter by')
self.parser.set_usage("usage: %prog search [<search_term>] [--galaxy-tags <galaxy_tag1,galaxy_tag2>] [--platforms platform]")
self.parser.add_option('--author', dest='author',
help='GitHub username')
self.parser.set_usage("usage: %prog search [searchterm1 searchterm2] [--galaxy-tags galaxy_tag1,galaxy_tag2] [--platforms platform1,platform2] [--author username]")
elif self.action == "setup":
self.parser.set_usage("usage: %prog setup [options] source github_user github_repo secret" +
u'\n\n' + "Create an integration with travis.")
self.parser.add_option('-r', '--remove', dest='remove_id', default=None,
help='Remove the integration matching the provided ID value. Use --list to see ID values.')
self.parser.add_option('-l', '--list', dest="setup_list", action='store_true', default=False,
help='List all of your integrations.')
# options that apply to more than one action
if self.action != "init":
if not self.action in ("config","import","init","login","setup"):
self.parser.add_option('-p', '--roles-path', dest='roles_path', default=C.DEFAULT_ROLES_PATH,
help='The path to the directory containing your roles. '
'The default is the roles_path configured in your '
'ansible.cfg file (/etc/ansible/roles if not configured)')
if self.action in ("info","init","install","search"):
self.parser.add_option('-s', '--server', dest='api_server', default="https://galaxy.ansible.com",
if self.action in ("import","info","init","install","login","search","setup","delete"):
self.parser.add_option('-s', '--server', dest='api_server', default=C.GALAXY_SERVER,
help='The API server destination')
self.parser.add_option('-c', '--ignore-certs', action='store_false', dest='validate_certs', default=True,
help='Ignore SSL certificate validation errors.')
@ -112,23 +173,25 @@ class GalaxyCLI(CLI):
self.parser.add_option('-f', '--force', dest='force', action='store_true', default=False,
help='Force overwriting an existing role')
# get options, args and galaxy object
self.options, self.args =self.parser.parse_args(self.args[1:])
display.verbosity = self.options.verbosity
self.galaxy = Galaxy(self.options)
if self.action:
# get options, args and galaxy object
self.options, self.args =self.parser.parse_args()
display.verbosity = self.options.verbosity
self.galaxy = Galaxy(self.options)
return True
def run(self):
if not self.action:
return True
super(GalaxyCLI, self).run()
# if not offline, get connect to galaxy api
if self.action in ("info","install", "search") or (self.action == 'init' and not self.options.offline):
api_server = self.options.api_server
self.api = GalaxyAPI(self.galaxy, api_server)
if not self.api:
raise AnsibleError("The API server (%s) is not responding, please try again later." % api_server)
if self.action in ("import","info","install","search","login","setup","delete") or \
(self.action == 'init' and not self.options.offline):
self.api = GalaxyAPI(self.galaxy)
self.execute()
@ -188,7 +251,7 @@ class GalaxyCLI(CLI):
"however it will reset any main.yml files that may have\n"
"been modified there already." % role_path)
# create the default README.md
# create default README.md
if not os.path.exists(role_path):
os.makedirs(role_path)
readme_path = os.path.join(role_path, "README.md")
@ -196,9 +259,16 @@ class GalaxyCLI(CLI):
f.write(self.galaxy.default_readme)
f.close()
# create default .travis.yml
travis = Environment().from_string(self.galaxy.default_travis).render()
f = open(os.path.join(role_path, '.travis.yml'), 'w')
f.write(travis)
f.close()
for dir in GalaxyRole.ROLE_DIRS:
dir_path = os.path.join(init_path, role_name, dir)
main_yml_path = os.path.join(dir_path, 'main.yml')
# create the directory if it doesn't exist already
if not os.path.exists(dir_path):
os.makedirs(dir_path)
@ -234,6 +304,20 @@ class GalaxyCLI(CLI):
f.write(rendered_meta)
f.close()
pass
elif dir == "tests":
# create tests/test.yml
inject = dict(
role_name = role_name
)
playbook = Environment().from_string(self.galaxy.default_test).render(inject)
f = open(os.path.join(dir_path, 'test.yml'), 'w')
f.write(playbook)
f.close()
# create tests/inventory
f = open(os.path.join(dir_path, 'inventory'), 'w')
f.write('localhost')
f.close()
elif dir not in ('files','templates'):
# just write a (mostly) empty YAML file for main.yml
f = open(main_yml_path, 'w')
@ -325,7 +409,7 @@ class GalaxyCLI(CLI):
for role in required_roles:
role = RoleRequirement.role_yaml_parse(role)
display.debug('found role %s in yaml file' % str(role))
display.vvv('found role %s in yaml file' % str(role))
if 'name' not in role and 'scm' not in role:
raise AnsibleError("Must specify name or src for role")
roles_left.append(GalaxyRole(self.galaxy, **role))
@ -348,7 +432,7 @@ class GalaxyCLI(CLI):
roles_left.append(GalaxyRole(self.galaxy, rname.strip()))
for role in roles_left:
display.debug('Installing role %s ' % role.name)
display.vvv('Installing role %s ' % role.name)
# query the galaxy API for the role data
if role.install_info is not None and not force:
@ -458,21 +542,189 @@ class GalaxyCLI(CLI):
return 0
def execute_search(self):
page_size = 1000
search = None
if len(self.args) > 1:
raise AnsibleOptionsError("At most a single search term is allowed.")
elif len(self.args) == 1:
search = self.args.pop()
if len(self.args):
terms = []
for i in range(len(self.args)):
terms.append(self.args.pop())
search = '+'.join(terms)
response = self.api.search_roles(search, self.options.platforms, self.options.tags)
if not search and not self.options.platforms and not self.options.tags and not self.options.author:
raise AnsibleError("Invalid query. At least one search term, platform, galaxy tag or author must be provided.")
if 'count' in response:
display.display("Found %d roles matching your search:\n" % response['count'])
response = self.api.search_roles(search, platforms=self.options.platforms,
tags=self.options.tags, author=self.options.author, page_size=page_size)
if response['count'] == 0:
display.display("No roles match your search.", color="yellow")
return True
data = ''
if 'results' in response:
for role in response['results']:
data += self._display_role_info(role)
if response['count'] > page_size:
data += ("Found %d roles matching your search. Showing first %s.\n" % (response['count'], page_size))
else:
data += ("Found %d roles matching your search:\n" % response['count'])
max_len = []
for role in response['results']:
max_len.append(len(role['username'] + '.' + role['name']))
name_len = max(max_len)
format_str = " %%-%ds %%s\n" % name_len
data +='\n'
data += (format_str % ("Name", "Description"))
data += (format_str % ("----", "-----------"))
for role in response['results']:
data += (format_str % (role['username'] + '.' + role['name'],role['description']))
self.pager(data)
return True
def execute_login(self):
"""
Verify user's identify via Github and retreive an auth token from Galaxy.
"""
# Authenticate with github and retrieve a token
if self.options.token is None:
login = GalaxyLogin(self.galaxy)
github_token = login.create_github_token()
else:
github_token = self.options.token
galaxy_response = self.api.authenticate(github_token)
if self.options.token is None:
# Remove the token we created
login.remove_github_token()
# Store the Galaxy token
token = GalaxyToken()
token.set(galaxy_response['token'])
display.display("Succesfully logged into Galaxy as %s" % galaxy_response['username'])
return 0
def execute_import(self):
"""
Import a role into Galaxy
"""
colors = {
'INFO': 'normal',
'WARNING': 'yellow',
'ERROR': 'red',
'SUCCESS': 'green',
'FAILED': 'red'
}
if len(self.args) < 2:
raise AnsibleError("Expected a github_username and github_repository. Use --help.")
github_repo = self.args.pop()
github_user = self.args.pop()
if self.options.check_status:
task = self.api.get_import_task(github_user=github_user, github_repo=github_repo)
else:
# Submit an import request
task = self.api.create_import_task(github_user, github_repo, reference=self.options.reference)
if len(task) > 1:
# found multiple roles associated with github_user/github_repo
display.display("WARNING: More than one Galaxy role associated with Github repo %s/%s." % (github_user,github_repo),
color='yellow')
display.display("The following Galaxy roles are being updated:" + u'\n', color='yellow')
for t in task:
display.display('%s.%s' % (t['summary_fields']['role']['namespace'],t['summary_fields']['role']['name']), color='yellow')
display.display(u'\n' + "To properly namespace this role, remove each of the above and re-import %s/%s from scratch" % (github_user,github_repo),
color='yellow')
return 0
# found a single role as expected
display.display("Successfully submitted import request %d" % task[0]['id'])
if not self.options.wait:
display.display("Role name: %s" % task[0]['summary_fields']['role']['name'])
display.display("Repo: %s/%s" % (task[0]['github_user'],task[0]['github_repo']))
if self.options.check_status or self.options.wait:
# Get the status of the import
msg_list = []
finished = False
while not finished:
task = self.api.get_import_task(task_id=task[0]['id'])
for msg in task[0]['summary_fields']['task_messages']:
if msg['id'] not in msg_list:
display.display(msg['message_text'], color=colors[msg['message_type']])
msg_list.append(msg['id'])
if task[0]['state'] in ['SUCCESS', 'FAILED']:
finished = True
else:
time.sleep(10)
return 0
def execute_setup(self):
"""
Setup an integration from Github or Travis
"""
if self.options.setup_list:
# List existing integration secrets
secrets = self.api.list_secrets()
if len(secrets) == 0:
# None found
display.display("No integrations found.")
return 0
display.display(u'\n' + "ID Source Repo", color="green")
display.display("---------- ---------- ----------", color="green")
for secret in secrets:
display.display("%-10s %-10s %s/%s" % (secret['id'], secret['source'], secret['github_user'],
secret['github_repo']),color="green")
return 0
if self.options.remove_id:
# Remove a secret
self.api.remove_secret(self.options.remove_id)
display.display("Secret removed. Integrations using this secret will not longer work.", color="green")
return 0
if len(self.args) < 4:
raise AnsibleError("Missing one or more arguments. Expecting: source github_user github_repo secret")
return 0
secret = self.args.pop()
github_repo = self.args.pop()
github_user = self.args.pop()
source = self.args.pop()
resp = self.api.add_secret(source, github_user, github_repo, secret)
display.display("Added integration for %s %s/%s" % (resp['source'], resp['github_user'], resp['github_repo']))
return 0
def execute_delete(self):
"""
Delete a role from galaxy.ansible.com
"""
if len(self.args) < 2:
raise AnsibleError("Missing one or more arguments. Expected: github_user github_repo")
github_repo = self.args.pop()
github_user = self.args.pop()
resp = self.api.delete_role(github_user, github_repo)
if len(resp['deleted_roles']) > 1:
display.display("Deleted the following roles:")
display.display("ID User Name")
display.display("------ --------------- ----------")
for role in resp['deleted_roles']:
display.display("%-8s %-15s %s" % (role.id,role.namespace,role.name))
display.display(resp['status'])
return True

3
lib/ansible/constants.py
View file

@ -255,7 +255,8 @@ ACCELERATE_MULTI_KEY = get_config(p, 'accelerate', 'accelerate_multi_k
PARAMIKO_PTY = get_config(p, 'paramiko_connection', 'pty', 'ANSIBLE_PARAMIKO_PTY', True, boolean=True)
# galaxy related
DEFAULT_GALAXY_URI = get_config(p, 'galaxy', 'server_uri', 'ANSIBLE_GALAXY_SERVER_URI', 'https://galaxy.ansible.com')
GALAXY_SERVER = get_config(p, 'galaxy', 'server', 'ANSIBLE_GALAXY_SERVER', 'https://galaxy.ansible.com')
GALAXY_IGNORE_CERTS = get_config(p, 'galaxy', 'ignore_certs', 'ANSIBLE_GALAXY_IGNORE', False, boolean=True)
# this can be configured to blacklist SCMS but cannot add new ones unless the code is also updated
GALAXY_SCMS = get_config(p, 'galaxy', 'scms', 'ANSIBLE_GALAXY_SCMS', 'git, hg', islist=True)

2
lib/ansible/galaxy/__init__.py
View file

@ -52,6 +52,8 @@ class Galaxy(object):
#TODO: move to getter for lazy loading
self.default_readme = self._str_from_data_file('readme')
self.default_meta = self._str_from_data_file('metadata_template.j2')
self.default_test = self._str_from_data_file('test_playbook.j2')
self.default_travis = self._str_from_data_file('travis.j2')
def add_role(self, role):
self.roles[role.name] = role

205
lib/ansible/galaxy/api.py
View file

@ -25,11 +25,15 @@ from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import json
import urllib
from urllib2 import quote as urlquote, HTTPError
from urlparse import urlparse
import ansible.constants as C
from ansible.errors import AnsibleError
from ansible.module_utils.urls import open_url
from ansible.galaxy.token import GalaxyToken
try:
from __main__ import display
@ -43,45 +47,113 @@ class GalaxyAPI(object):
SUPPORTED_VERSIONS = ['v1']
def __init__(self, galaxy, api_server):
def __init__(self, galaxy):
self.galaxy = galaxy
self.token = GalaxyToken()
self._api_server = C.GALAXY_SERVER
self._validate_certs = C.GALAXY_IGNORE_CERTS
try:
urlparse(api_server, scheme='https')
except:
raise AnsibleError("Invalid server API url passed: %s" % api_server)
# set validate_certs
if galaxy.options.validate_certs == False:
self._validate_certs = False
display.vvv('Check for valid certs: %s' % self._validate_certs)
server_version = self.get_server_api_version('%s/api/' % (api_server))
if not server_version:
raise AnsibleError("Could not retrieve server API version: %s" % api_server)
# set the API server
if galaxy.options.api_server != C.GALAXY_SERVER:
self._api_server = galaxy.options.api_server
display.vvv("Connecting to galaxy_server: %s" % self._api_server)
server_version = self.get_server_api_version()
if server_version in self.SUPPORTED_VERSIONS:
self.baseurl = '%s/api/%s' % (api_server, server_version)
self.baseurl = '%s/api/%s' % (self._api_server, server_version)
self.version = server_version # for future use
display.vvvvv("Base API: %s" % self.baseurl)
display.vvv("Base API: %s" % self.baseurl)
else:
raise AnsibleError("Unsupported Galaxy server API version: %s" % server_version)
def get_server_api_version(self, api_server):
def __auth_header(self):
token = self.token.get()
if token is None:
raise AnsibleError("No access token. You must first use login to authenticate and obtain an access token.")
return {'Authorization': 'Token ' + token}
def __call_galaxy(self, url, args=None, headers=None, method=None):
if args and not headers:
headers = self.__auth_header()
try:
display.vvv(url)
resp = open_url(url, data=args, validate_certs=self._validate_certs, headers=headers, method=method)
data = json.load(resp)
except HTTPError as e:
res = json.load(e)
raise AnsibleError(res['detail'])
return data
@property
def api_server(self):
return self._api_server
@property
def validate_certs(self):
return self._validate_certs
def get_server_api_version(self):
"""
Fetches the Galaxy API current version to ensure
the API server is up and reachable.
"""
#TODO: fix galaxy server which returns current_version path (/api/v1) vs actual version (v1)
# also should set baseurl using supported_versions which has path
return 'v1'
try:
data = json.load(open_url(api_server, validate_certs=self.galaxy.options.validate_certs))
return data.get("current_version", 'v1')
except Exception:
# TODO: report error
return None
url = '%s/api/' % self._api_server
data = json.load(open_url(url, validate_certs=self._validate_certs))
return data['current_version']
except Exception as e:
raise AnsibleError("The API server (%s) is not responding, please try again later." % url)
def authenticate(self, github_token):
"""
Retrieve an authentication token
"""
url = '%s/tokens/' % self.baseurl
args = urllib.urlencode({"github_token": github_token})
resp = open_url(url, data=args, validate_certs=self._validate_certs, method="POST")
data = json.load(resp)
return data
def create_import_task(self, github_user, github_repo, reference=None):
"""
Post an import request
"""
url = '%s/imports/' % self.baseurl
args = urllib.urlencode({
"github_user": github_user,
"github_repo": github_repo,
"github_reference": reference if reference else ""
})
data = self.__call_galaxy(url, args=args)
if data.get('results', None):
return data['results']
return data
def get_import_task(self, task_id=None, github_user=None, github_repo=None):
"""
Check the status of an import task.
"""
url = '%s/imports/' % self.baseurl
if not task_id is None:
url = "%s?id=%d" % (url,task_id)
elif not github_user is None and not github_repo is None:
url = "%s?github_user=%s&github_repo=%s" % (url,github_user,github_repo)
else:
raise AnsibleError("Expected task_id or github_user and github_repo")
data = self.__call_galaxy(url)
return data['results']
def lookup_role_by_name(self, role_name, notify=True):
"""
Find a role by name
Find a role by name.
"""
role_name = urlquote(role_name)
@ -92,18 +164,12 @@ class GalaxyAPI(object):
if notify:
display.display("- downloading role '%s', owned by %s" % (role_name, user_name))
except:
raise AnsibleError("- invalid role name (%s). Specify role as format: username.rolename" % role_name)
raise AnsibleError("Invalid role name (%s). Specify role as format: username.rolename" % role_name)
url = '%s/roles/?owner__username=%s&name=%s' % (self.baseurl, user_name, role_name)
display.vvvv("- %s" % (url))
try:
data = json.load(open_url(url, validate_certs=self.galaxy.options.validate_certs))
if len(data["results"]) != 0:
return data["results"][0]
except:
# TODO: report on connection/availability errors
pass
data = self.__call_galaxy(url)
if len(data["results"]) != 0:
return data["results"][0]
return None
def fetch_role_related(self, related, role_id):
@ -114,13 +180,12 @@ class GalaxyAPI(object):
try:
url = '%s/roles/%d/%s/?page_size=50' % (self.baseurl, int(role_id), related)
data = json.load(open_url(url, validate_certs=self.galaxy.options.validate_certs))
data = self.__call_galaxy(url)
results = data['results']
done = (data.get('next', None) is None)
while not done:
url = '%s%s' % (self.baseurl, data['next'])
display.display(url)
data = json.load(open_url(url, validate_certs=self.galaxy.options.validate_certs))
data = self.__call_galaxy(url)
results += data['results']
done = (data.get('next', None) is None)
return results
@ -131,10 +196,9 @@ class GalaxyAPI(object):
"""
Fetch the list of items specified.
"""
try:
url = '%s/%s/?page_size' % (self.baseurl, what)
data = json.load(open_url(url, validate_certs=self.galaxy.options.validate_certs))
data = self.__call_galaxy(url)
if "results" in data:
results = data['results']
else:
@ -144,41 +208,64 @@ class GalaxyAPI(object):
done = (data.get('next', None) is None)
while not done:
url = '%s%s' % (self.baseurl, data['next'])
display.display(url)
data = json.load(open_url(url, validate_certs=self.galaxy.options.validate_certs))
data = self.__call_galaxy(url)
results += data['results']
done = (data.get('next', None) is None)
return results
except Exception as error:
raise AnsibleError("Failed to download the %s list: %s" % (what, str(error)))
def search_roles(self, search, platforms=None, tags=None):
def search_roles(self, search, **kwargs):
search_url = self.baseurl + '/roles/?page=1'
search_url = self.baseurl + '/search/roles/?'
if search:
search_url += '&search=' + urlquote(search)
search_url += '&autocomplete=' + urlquote(search)
if tags is None:
tags = []
elif isinstance(tags, basestring):
tags = kwargs.get('tags',None)
platforms = kwargs.get('platforms', None)
page_size = kwargs.get('page_size', None)
author = kwargs.get('author', None)
if tags and isinstance(tags, basestring):
tags = tags.split(',')
for tag in tags:
search_url += '&chain__tags__name=' + urlquote(tag)
if platforms is None:
platforms = []
elif isinstance(platforms, basestring):
search_url += '&tags_autocomplete=' + '+'.join(tags)
if platforms and isinstance(platforms, basestring):
platforms = platforms.split(',')
search_url += '&platforms_autocomplete=' + '+'.join(platforms)
for plat in platforms:
search_url += '&chain__platforms__name=' + urlquote(plat)
display.debug("Executing query: %s" % search_url)
try:
data = json.load(open_url(search_url, validate_certs=self.galaxy.options.validate_certs))
except HTTPError as e:
raise AnsibleError("Unsuccessful request to server: %s" % str(e))
if page_size:
search_url += '&page_size=%s' % page_size
if author:
search_url += '&username_autocomplete=%s' % author
data = self.__call_galaxy(search_url)
return data
def add_secret(self, source, github_user, github_repo, secret):
url = "%s/notification_secrets/" % self.baseurl
args = urllib.urlencode({
"source": source,
"github_user": github_user,
"github_repo": github_repo,
"secret": secret
})
data = self.__call_galaxy(url, args=args)
return data
def list_secrets(self):
url = "%s/notification_secrets" % self.baseurl
data = self.__call_galaxy(url, headers=self.__auth_header())
return data
def remove_secret(self, secret_id):
url = "%s/notification_secrets/%s/" % (self.baseurl, secret_id)
data = self.__call_galaxy(url, headers=self.__auth_header(), method='DELETE')
return data
def delete_role(self, github_user, github_repo):
url = "%s/removerole/?github_user=%s&github_repo=%s" % (self.baseurl,github_user,github_repo)
data = self.__call_galaxy(url, headers=self.__auth_header(), method='DELETE')
return data

14
lib/ansible/galaxy/data/metadata_template.j2
View file

@ -2,9 +2,11 @@ galaxy_info:
author: {{ author }}
description: {{description}}
company: {{ company }}
# If the issue tracker for your role is not on github, uncomment the
# next line and provide a value
# issue_tracker_url: {{ issue_tracker_url }}
# Some suggested licenses:
# - BSD (default)
# - MIT
@ -13,7 +15,17 @@ galaxy_info:
# - Apache
# - CC-BY
license: {{ license }}
min_ansible_version: {{ min_ansible_version }}
# Optionally specify the branch Galaxy will use when accessing the GitHub
# repo for this role. During role install, if no tags are available,
# Galaxy will use this branch. During import Galaxy will access files on
# this branch. If travis integration is cofigured, only notification for this
# branch will be accepted. Otherwise, in all cases, the repo's default branch
# (usually master) will be used.
#github_branch:
#
# Below are all platforms currently available. Just uncomment
# the ones that apply to your role. If you don't see your
@ -28,6 +40,7 @@ galaxy_info:
# - {{ version }}
{%- endfor %}
{%- endfor %}
galaxy_tags: []
# List tags for your role here, one per line. A tag is
# a keyword that describes and categorizes the role.
@ -36,6 +49,7 @@ galaxy_info:
#
# NOTE: A tag is limited to a single word comprised of
# alphanumeric characters. Maximum 20 tags per role.
dependencies: []
# List your role dependencies here, one per line.
# Be sure to remove the '[]' above if you add dependencies

5
lib/ansible/galaxy/data/test_playbook.j2 Normal file
View file

@ -0,0 +1,5 @@
---
- hosts: localhost
remote_user: root
roles:
- {{ role_name }}

29
lib/ansible/galaxy/data/travis.j2 Normal file
View file

@ -0,0 +1,29 @@
---
language: python
python: "2.7"
# Use the new container infrastructure
sudo: false
# Install ansible
addons:
apt:
packages:
- python-pip
install:
# Install ansible
- pip install ansible
# Check ansible version
- ansible --version
# Create ansible.cfg with correct roles_path
- printf '[defaults]\nroles_path=../' >ansible.cfg
script:
# Basic role syntax check
- ansible-playbook tests/test.yml -i tests/inventory --syntax-check
notifications:
webhooks: https://galaxy.ansible.com/api/v1/notifications/

113
lib/ansible/galaxy/login.py Normal file
View file

@ -0,0 +1,113 @@
#!/usr/bin/env python
########################################################################
#
# (C) 2015, Chris Houseknecht <chouse@ansible.com>
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible. If not, see <http://www.gnu.org/licenses/>.
#
########################################################################
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import getpass
import json
import urllib
from urllib2 import quote as urlquote, HTTPError
from urlparse import urlparse
from ansible.errors import AnsibleError, AnsibleOptionsError
from ansible.module_utils.urls import open_url
from ansible.utils.color import stringc
try:
from __main__ import display
except ImportError:
from ansible.utils.display import Display
display = Display()
class GalaxyLogin(object):
''' Class to handle authenticating user with Galaxy API prior to performing CUD operations '''
GITHUB_AUTH = 'https://api.github.com/authorizations'
def __init__(self, galaxy, github_token=None):
self.galaxy = galaxy
self.github_username = None
self.github_password = None
if github_token == None:
self.get_credentials()
def get_credentials(self):
display.display(u'\n\n' + "We need your " + stringc("Github login",'bright cyan') +
" to identify you.", screen_only=True)
display.display("This information will " + stringc("not be sent to Galaxy",'bright cyan') +
", only to " + stringc("api.github.com.","yellow"), screen_only=True)
display.display("The password will not be displayed." + u'\n\n', screen_only=True)
display.display("Use " + stringc("--github-token",'yellow') +
" if you do not want to enter your password." + u'\n\n', screen_only=True)
try:
self.github_username = raw_input("Github Username: ")
except:
pass
try:
self.github_password = getpass.getpass("Password for %s: " % self.github_username)
except:
pass
if not self.github_username or not self.github_password:
raise AnsibleError("Invalid Github credentials. Username and password are required.")
def remove_github_token(self):
'''
If for some reason an ansible-galaxy token was left from a prior login, remove it. We cannot
retrieve the token after creation, so we are forced to create a new one.
'''
try:
tokens = json.load(open_url(self.GITHUB_AUTH, url_username=self.github_username,
url_password=self.github_password, force_basic_auth=True,))
except HTTPError as e:
res = json.load(e)
raise AnsibleError(res['message'])
for token in tokens:
if token['note'] == 'ansible-galaxy login':
display.vvvvv('removing token: %s' % token['token_last_eight'])
try:
open_url('https://api.github.com/authorizations/%d' % token['id'], url_username=self.github_username,
url_password=self.github_password, method='DELETE', force_basic_auth=True,)
except HTTPError as e:
res = json.load(e)
raise AnsibleError(res['message'])
def create_github_token(self):
'''
Create a personal authorization token with a note of 'ansible-galaxy login'
'''
self.remove_github_token()
args = json.dumps({"scopes":["public_repo"], "note":"ansible-galaxy login"})
try:
data = json.load(open_url(self.GITHUB_AUTH, url_username=self.github_username,
url_password=self.github_password, force_basic_auth=True, data=args))
except HTTPError as e:
res = json.load(e)
raise AnsibleError(res['message'])
return data['token']

10
lib/ansible/galaxy/role.py
View file

@ -46,7 +46,7 @@ class GalaxyRole(object):
SUPPORTED_SCMS = set(['git', 'hg'])
META_MAIN = os.path.join('meta', 'main.yml')
META_INSTALL = os.path.join('meta', '.galaxy_install_info')
ROLE_DIRS = ('defaults','files','handlers','meta','tasks','templates','vars')
ROLE_DIRS = ('defaults','files','handlers','meta','tasks','templates','vars','tests')
def __init__(self, galaxy, name, src=None, version=None, scm=None, path=None):
@ -198,10 +198,10 @@ class GalaxyRole(object):
role_data = self.src
tmp_file = self.fetch(role_data)
else:
api = GalaxyAPI(self.galaxy, self.options.api_server)
api = GalaxyAPI(self.galaxy)
role_data = api.lookup_role_by_name(self.src)
if not role_data:
raise AnsibleError("- sorry, %s was not found on %s." % (self.src, self.options.api_server))
raise AnsibleError("- sorry, %s was not found on %s." % (self.src, api.api_server))
role_versions = api.fetch_role_related('versions', role_data['id'])
if not self.version:
@ -213,8 +213,10 @@ class GalaxyRole(object):
loose_versions = [LooseVersion(a.get('name',None)) for a in role_versions]
loose_versions.sort()
self.version = str(loose_versions[-1])
elif role_data.get('github_branch', None):
self.version = role_data['github_branch']
else:
self.version = 'master'
self.version = 'master'
elif self.version != 'master':
if role_versions and self.version not in [a.get('name', None) for a in role_versions]:
raise AnsibleError("- the specified version (%s) of %s was not found in the list of available versions (%s)." % (self.version, self.name, role_versions))

67
lib/ansible/galaxy/token.py Normal file
View file

@ -0,0 +1,67 @@
#!/usr/bin/env python
########################################################################
#
# (C) 2015, Chris Houseknecht <chouse@ansible.com>
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible. If not, see <http://www.gnu.org/licenses/>.
#
########################################################################
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import os
import yaml
from stat import *
try:
from __main__ import display
except ImportError:
from ansible.utils.display import Display
display = Display()
class GalaxyToken(object):
''' Class to storing and retrieving token in ~/.ansible_galaxy '''
def __init__(self):
self.file = os.path.expanduser("~") + '/.ansible_galaxy'
self.config = yaml.safe_load(self.__open_config_for_read())
if not self.config:
self.config = {}
def __open_config_for_read(self):
if os.path.isfile(self.file):
display.vvv('Opened %s' % self.file)
return open(self.file, 'r')
# config.yml not found, create and chomd u+rw
f = open(self.file,'w')
f.close()
os.chmod(self.file,S_IRUSR|S_IWUSR) # owner has +rw
display.vvv('Created %s' % self.file)
return open(self.file, 'r')
def set(self, token):
self.config['token'] = token
self.save()
def get(self):
return self.config.get('token', None)
def save(self):
with open(self.file,'w') as f:
yaml.safe_dump(self.config,f,default_flow_style=False)