Merge branch 'role_dependencies' of git://github.com/jimi1283/ansible into jimi1283-role_dependencies

docsite/latest/rst/playbooks.rst

@@ -466,12 +466,14 @@ Example project structure::
          tasks/
          handlers/
          vars/
+         meta/
        webservers/
          files/
          templates/
          tasks/
          handlers/
          vars/
+         meta/
 
 In a playbook, it would look like this::
 
@@ -486,10 +488,14 @@ This designates the following behaviors, for each role 'x':
 - If roles/x/tasks/main.yml exists, tasks listed therein will be added to the play
 - If roles/x/handlers/main.yml exists, handlers listed therein will be added to the play
 - If roles/x/vars/main.yml exists, variables listed therein will be added to the play
+- If roles/x/meta/main.yml exists, any role dependencies listed therein will be added to the list of roles
 - Any copy tasks can reference files in roles/x/files/ without having to path them relatively or absolutely
 - Any script tasks can reference scripts in roles/x/files/ without having to path them relatively or absolutely
 - Any template tasks can reference files in roles/x/templates/ without having to path them relatively or absolutely
 
+.. note::
+   Role dependencies are discussed below.
+
 If any files are not present, they are just ignored.  So it's ok to not have a 'vars/' subdirectory for the role,
 for instance.
 
@@ -544,6 +550,99 @@ If you want to define certain tasks to happen before AND after roles are applied
    be sure to also tag your pre_tasks and post_tasks and pass those along as well, especially if the pre
    and post tasks are used for monitoring outage window control or load balancing.
 
+Role Dependencies
+`````````````````
+
+.. versionadded: 1.3
+
+Role dependencies allow you to include other roles within your role, so that you no longer
+have to specify them at the top level. As noted above, role dependencies are stored in the
+`meta/main.yml` file contained within the role directory. This file should contain the following::
+
+    ---
+    dependencies:
+    - { role: foo, x: 1 }
+    - { role: bar, y: 2 }
+    - { role: baz, z: 3 }
+
+Role dependencies can also be specified as a full path::
+
+    ---
+    dependencies:
+    - { role: '/path/to/common/roles/foo', x: 1 }
+
+Roles dependencies are always executed before the role that includes them. For example, given the following
+list of dependant roles::
+
+    - car
+      - wheel
+        - tire
+        - brake
+
+The roles would be executed in the order: tire -> brake -> wheel -> car.
+
+Role dependencies may be included more than once. Continuing the above example, the car role could 
+add dependencies as follows::
+
+    ---
+    dependencies:
+    - { role: wheel, n: 1 }
+    - { role: wheel, n: 2 }
+    - { role: wheel, n: 3 }
+    - { role: wheel, n: 4 }
+
+Which would result in the following dependency tree::
+
+    - car
+      - wheel (n=1)
+        - tire (n=1)
+        - brake (n=1)
+      - wheel (n=2)
+        - tire (n=2)
+        - brake (n=2)
+      - wheel (n=3)
+        - tire (n=3)
+        - brake (n=3)
+      - wheel (n=4)
+        - tire (n=4)
+        - brake (n=4)
+
+And the order of execution would be tire(n=1) -> brake(n=1) -> wheel(n=1) -> tire(n=2) -> brake(n=2) -> wheel(n=2) -> ... -> car.
+
+.. note::
+   Variable inheritance and scope are detailed below.
+
+Role Variable Scope and Precedence
+``````````````````````````````````
+
+There are two rules governing variable scope when it comes to roles and dependencies.
+
+1. Variables listed in vars/ files are loaded into the role and also into the global list of variables.
+
+This means that if two roles define the same variable name, the last one to be included will be the 
+one that sets the variable at the global level. These variables also override whatever may be set in group
+or host vars files, since inventory variables have the lowest priority.
+
+This allows roles to share variables with other roles that it doesn't know about, and means variables from 
+parent roles will override any that are set at a lower level. Given the car/wheel example above, if the 
+`tire` role sets `x: 1` in its vars/main.yml while the `wheel` roles sets `x: 2`, both roles will see 
+`x: 2` (as will the brake role). This allows parent roles to override variables defined in dependant classes, 
+for instance if you wanted to override the http_port setting in a web server role. 
+
+If you wish to avoid this behavior, make sure the variables in your roles have unique names instead of something 
+generic like `port`.
+
+2. Variables given when including/depending a role override variables in vars/main.yml
+
+This means that if you include a role (or add it to a list of dependencies) while setting a variable, 
+that variable value will be the one that role (and any dependant roles) will see.
+
+For example, given the car/wheel example again, if the car adds the wheel role as a dependency as follows::
+
+    - { role: wheel, x: 100 }
+
+Then the wheel, tire, and brake roles will all see `x: 100` no matter what is set in the vars files for each role.
+
 Executing A Playbook
 ````````````````````
 

docsite/latest/rst/playbooks2.rst

@@ -1101,7 +1101,7 @@ Which of course means that, though more verbose, this is also legal syntax::
 Local Facts (Facts.d)
 `````````````````````
 
-.. version_added:: 1.3
+.. versionadded:: 1.3
 
 As discussed in the playbooks chapter, Ansible facts are a way of getting data about remote systems for use in playbook variables.
 Usually these are discovered automatically by the 'setup' module in Ansible. Users can also write custom facts modules, as described

lib/ansible/playbook/play.py

@@ -119,9 +119,71 @@ class Play(object):
 
     # *************************************************
 
-    def _load_roles(self, roles, ds):
+    def _get_role_path(self, role):
+        """
+        Returns the path on disk to the directory containing
+        the role directories like tasks, templates, etc. Also 
+        returns any variables that were included with the role
+        """
+        orig_path = template(self.basedir,role,self.vars)
+
+        role_vars = {}
+        if type(orig_path) == dict:
+            # what, not a path?
+            role_name = orig_path.get('role', None)
+            if role_name is None:
+                raise errors.AnsibleError("expected a role name in dictionary: %s" % orig_path)
+            role_vars = orig_path
+            orig_path = role_name
 
+        path = utils.path_dwim(self.basedir, os.path.join('roles', orig_path))
+        if not os.path.isdir(path) and not orig_path.startswith(".") and not orig_path.startswith("/"):
+            path2 = utils.path_dwim(self.basedir, orig_path)
+            if not os.path.isdir(path2):
+                raise errors.AnsibleError("cannot find role in %s or %s" % (path, path2))
+            path = path2
+        elif not os.path.isdir(path):
+            raise errors.AnsibleError("cannot find role in %s" % (path))
 
+        return (path, role_vars)
+
+    def _build_role_dependencies(self, roles, dep_stack, passed_vars={}, level=0):
+        # this number is arbitrary, but it seems sane
+        if level > 20:
+            raise errors.AnsibleError("too many levels of recursion while resolving role dependencies")
+        for role in roles:
+            role_path,role_vars = self._get_role_path(role)
+            # the meta directory contains the yaml that should
+            # hold the list of dependencies (if any)
+            meta = self._resolve_main(utils.path_dwim(self.basedir, os.path.join(role_path, 'meta')))
+            if os.path.isfile(meta):
+                data = utils.parse_yaml_from_file(meta)
+                if data:
+                    dependencies = data.get('dependencies',[])
+                    for dep in dependencies:
+                        (dep_path,dep_vars) = self._get_role_path(dep)
+                        vars = self._resolve_main(utils.path_dwim(self.basedir, os.path.join(dep_path, 'vars')))
+                        vars_data = {}
+                        if os.path.isfile(vars):
+                            vars_data = utils.parse_yaml_from_file(vars)
+                        dep_vars.update(role_vars)
+                        for k in passed_vars.keys():
+                            if not k in dep_vars:
+                                dep_vars[k] = passed_vars[k]
+                        for k in vars_data.keys():
+                            if not k in dep_vars:
+                                dep_vars[k] = vars_data[k]
+                        if 'role' in dep_vars:
+                            del dep_vars['role']
+                        self._build_role_dependencies([dep], dep_stack, passed_vars=dep_vars, level=level+1)
+                        dep_stack.append([dep,dep_path,dep_vars])
+                    # only add the current role when we're at the top level,
+                    # otherwise we'll end up in a recursive loop 
+                    if level == 0:
+                        dep_stack.append([role,role_path,role_vars])
+        return dep_stack
+
+    def _load_roles(self, roles, ds):
         # a role is a name that auto-includes the following if they exist
         #    <rolename>/tasks/main.yml
         #    <rolename>/handlers/main.yml
@@ -147,52 +209,35 @@ class Play(object):
         # flush handlers after pre_tasks
         new_tasks.append(dict(meta='flush_handlers'))
 
-        # variables if the role was parameterized (i.e. given as a hash)
+        roles = self._build_role_dependencies(roles, [], self.vars)
-        has_dict = {}
-
-        for role_path in roles:
-            orig_path = template(self.basedir,role_path,self.vars)
-
-            if type(orig_path) == dict:
-                # what, not a path?
-                role_name = orig_path.get('role', None)
-                if role_name is None:
-                    raise errors.AnsibleError("expected a role name in dictionary: %s" % orig_path)
-                has_dict = orig_path
-                orig_path = role_name
 
+        for role,role_path,role_vars in roles:
             # special vars must be extracted from the dict to the included tasks
             special_keys = [ "sudo", "sudo_user", "when", "with_items" ]
             special_vars = {}
             for k in special_keys:
-                if k in has_dict:
+                if k in role_vars:
-                    special_vars[k] = has_dict[k]
+                    special_vars[k] = role_vars[k]
+
+            task_basepath    = utils.path_dwim(self.basedir, os.path.join(role_path, 'tasks'))
+            handler_basepath = utils.path_dwim(self.basedir, os.path.join(role_path, 'handlers'))
+            vars_basepath    = utils.path_dwim(self.basedir, os.path.join(role_path, 'vars'))
 
-            path = utils.path_dwim(self.basedir, os.path.join('roles', orig_path))
-            if not os.path.isdir(path) and not orig_path.startswith(".") and not orig_path.startswith("/"):
-                path2 = utils.path_dwim(self.basedir, orig_path)
-                if not os.path.isdir(path2):
-                    raise errors.AnsibleError("cannot find role in %s or %s" % (path, path2))
-                path = path2
-            elif not os.path.isdir(path):
-                raise errors.AnsibleError("cannot find role in %s" % (path))
-            task_basepath    = utils.path_dwim(self.basedir, os.path.join(path, 'tasks'))
-            handler_basepath = utils.path_dwim(self.basedir, os.path.join(path, 'handlers'))
-            vars_basepath    = utils.path_dwim(self.basedir, os.path.join(path, 'vars'))
             task      = self._resolve_main(task_basepath)
             handler   = self._resolve_main(handler_basepath)
             vars_file = self._resolve_main(vars_basepath)
-            library   = utils.path_dwim(self.basedir, os.path.join(path, 'library'))
+            library   = utils.path_dwim(self.basedir, os.path.join(role_path, 'library'))
+
             if not os.path.isfile(task) and not os.path.isfile(handler) and not os.path.isfile(vars_file) and not os.path.isdir(library):
-                raise errors.AnsibleError("found role at %s, but cannot find %s or %s or %s or %s" % (path, task, handler, vars_file, library))
+                raise errors.AnsibleError("found role at %s, but cannot find %s or %s or %s or %s" % (role_path, task, handler, vars_file, library))
             if os.path.isfile(task):
-                nt = dict(include=pipes.quote(task), vars=has_dict)
+                nt = dict(include=pipes.quote(task), vars=role_vars)
                 for k in special_keys:
                     if k in special_vars:
                         nt[k] = special_vars[k]
                 new_tasks.append(nt)
             if os.path.isfile(handler):
-                nt = dict(include=pipes.quote(handler), vars=has_dict)
+                nt = dict(include=pipes.quote(handler), vars=role_vars)
                 for k in special_keys:
                     if k in special_vars:
                         nt[k] = special_vars[k]
@@ -204,7 +249,6 @@ class Play(object):
 
         tasks      = ds.get('tasks', None)
         post_tasks = ds.get('post_tasks', None)
-
         handlers   = ds.get('handlers', None)
         vars_files = ds.get('vars_files', None)
 
@@ -223,8 +267,10 @@ class Play(object):
         new_tasks.extend(post_tasks)
         # flush handlers after post tasks
         new_tasks.append(dict(meta='flush_handlers'))
+
         new_handlers.extend(handlers)
         new_vars_files.extend(vars_files)
+
         ds['tasks'] = new_tasks
         ds['handlers'] = new_handlers
         ds['vars_files'] = new_vars_files