Sunday, 17 May 2015

TripleO Heat templates Part 3 - Cluster configuration, introduction/primer

In my previous two posts I covered an overview of TripleO template roles and groups, and specifics of how initial deployment of a node happens.  Today I'm planning to introduce the next step of the deployment process - taking the deployed groups of nodes, and configuring them to work together as clusters running the various OpenStack services encapsulated by each role.

This post will provide an introduction to the patterns and Heat features used to configure the groups of nodes, then in the next instalment I'll dig into the specifics of exactly what configuration takes place in the TripleO heat templates.


Wednesday, 13 May 2015

TripleO Heat templates Part 2 - Node initial deployment & config

In my previous post "TripleO Heat templates Part 1 - roles and groups", I provided an overview of the various TripleO roles, the way the role implementation is abstracted via provider resources, and how they are grouped and scaled via OS::Heat::ResourceGroup.

In this post, I'm aiming to dig into the next level of template implementation, specifically how a role is implemented behind the provider resource alias used in the top-level template.

I'm only going to cover one role type for now OS::TripleO::Controller, because the patterns described are directly applicable to all other role types.  I'm also going to focus on the puppet based implementation (because that's what I'm most familiar with), but again most concepts apply to the element/container/etc based implementations too.

Throughout this post, I'll be referring to templates in the tripleo-heat-templates repo, so if you haven't already, now might be a good time to clone that so you can follow along looking at the templates themselves.

Thursday, 7 May 2015

TripleO Heat templates Part 1 - Roles and Groups

This is the start of a series of posts aiming to de-construct the TripleO heat templates, explaining the abstractions that exist,and the heat features which enable them.

If you're not already a little familiar with ResourceGroups, "Provider Resources" used for template composition, and SoftwareConfig resources, it's probably not a bad idea to check out my previous posts on those topics, as well as our user guide and other documentation - TripleO makes heavy use of all of these features.

Overcloud "Roles"



TripleO typically refers to the deployed OpenStack cloud as the "overcloud", because the tools used to perform that deployment mirror those in the deployed cloud - e.g a small OpenStack is used to bootstrap and manage a bigger one (normally the small OpenStack is called either a "seed" or "undercloud", depending on your environment).

The definition of what is deployed in your overcloud exists in a number of Heat templates, with the top-level one defining a number of groups of different node types, or "roles".

  • Controller: Contains API services, e.g Keystone, Neutron, Heat, Glance, Ceilometer, Horizon, and the API parts of Nova, Cinder & Swift.  It can also optionally host the storage parts for Cinder, Swift and Ceph if these are not deployed separately (see below).
  • Compute: Contains the Nova Hypervisor components
  • BlockStorage: Contains the Cinder storage components (if not hosted on the Controller(s).
  • ObjectStorage: Contains the Swift storage components (if not hosted on the Controllers(s).
  • CephStorage: Contains the Ceph storage components (if not hosted on the Controllers(s).

Roles & resource types


Each of the roles (or node types), are mapped to a a type defined in the resource_registry in the environment passed to Heat.

So, for example, the "Controller" role is defined in the heat template as a type OS::TripleO::Controller, and similar aliases exist for all the other roles.

The resource registry maps this type alias to another heat template, which implements whatever is required to deploy one node with that role.

So to create a node type "OS::TripleO::Controller" Heat may create a stack based on the template in "puppet/controller-puppet.yaml", or some other implementation based on whatever mapping exists in the resource_registry.

This makes it very easy if you want to plug in some alternate implementation, while maintaining the top-level template interfaces and deployment topology.  For example, work is currently in-progress implementing an alternate implementation using docker containers, as an alternative to the existing puppet and element based impelementations.







Roles & ResourceGroups


Each of these roles may be independently scaled - because they are defined in an OS::Heat::ResourceGroup.  The minimum you can deploy is one "Controller" and one "Compute" node (some roles may be deployed with zero nodes in the group).

Here's an example of what that looks like in the top level "overcloud-without-mergepy" template (this is the name of the main template TripleO uses to deploy OpenStack, the "without-mergepy" part is historical and refers to an older, now deprecated, implementation.)

  Controller:
    type: OS::Heat::ResourceGroup
    properties:
      count: {get_param: ControllerCount}
      resource_def:
        type: OS::TripleO::Controller
        properties:
          AdminPassword: {get_param: AdminPassword}
          AdminToken: {get_param: AdminToken}

          ...


Here, you can see we've defined a group of OS::TripleO::Controller resources in an OS::Heat::ResourceGroup, and the number of nodes deployed is controlled via a template parameter, "ControllerCount", and similarly a number of template parameters are referenced to provide input properties to enable configuration of the deployed controller node (I've abbreviated the full list of properties).

This pattern is repeated for all roles, so building a specified number of nodes for a particular role (or adding/removing them via a stack-update), is as simple as passing a different number into Heat as a stack parameter :)

That's all, folks


That's all for today - hopefully it provides an overview of the top-level interfaces provided by the TripleO Heat templates, and illustrates the following:

  • There are clearly defined node "roles", containing the various parts of your OpenStack deployment
  • The patterns used to define and implement these roles are repeated, which helps understand the templates despite them being fairly large.
  • The implementation is modular, and abstractions exist which make implementing different "back end" implementations relatively simple.
  • Deployments can be easily scaled due to using Heat's ResourceGroup functionality.
In future instalments I'll dig further into the individual node implementations, ways to easily plug in site-specific additional configuration, and ways in which you can control and validate the deployments performed via TripleO.

Tuesday, 5 May 2015

Heat SoftwareConfig resources - primer/overview.

In this post, I'm going to provide an overview of Heat's Software Configuration resources, as a preface to digging in more detail into the structure of TripleO heat templates, which leverage SoftwareConfig functionality to install and configure the deployed OpenStack cloud.

Heat has supported SoftwareConfig and SoftwareDeployment resources since the Icehouse release, in an effort to provide flexible and non-opinionated abstractions which enable integration with existing software configuration tools and scripts.

Monday, 20 April 2015

Debugging TripleO Heat templates

Lately, I've been spending increasing amounts of time working with TripleO heat templates, and have noticed some recurring aspects of my workflow whilst debugging them which I thought may be worth sharing.

For the uninitiated, TripleO is an OpenStack deployment project, which aims to deploy and manage OpenStack using standard OpenStack API's.  In practice, this means using Nova and Ironic for baremetal node provisioning, and Heat to orchestrate the deployment and configuration of the nodes.

The TripleO heat templates, unlike most of the heat examples, are pretty complex.  They make extensive use of many "advanced" features, such as nested stacks, using provider resources via the environment and also many software config resources.

This makes TripleO a fairly daunting target to those wishing to debug and modify and/or debug the TripleO templates.

Fortunately TripleO templates, although large, have many repeated patterns, and good levels of abstraction and modularity.  Combined with some recently added heat interfaces, it becomes rapidly less daunting, as I will demonstrate in the worked example below:


Thursday, 11 September 2014

Using Heat ResourceGroup resources


This has come up a few times recently, so I wanted to share a howto showing how (and how not) to use the group resources in Heat, e.g OS::Heat::ResourceGroup and OS::Heat::AutoScalingGroup.

The key thing to remember when dealing with these resources is that they can multiply any number of resources (expressed as a heat stack), not just individual resources. This is a very cool feature when you get your head around it! :)

Lets go through a worked example, where we use ResourceGroup to create 5 identical servers, each with a cinder volume of the same size attached.

Wednesday, 16 April 2014

Heat auth model updates - part 2 "Stack Domain Users"

As promised, here's the second part of my updates on the Heat auth model, following on from part 1 describing our use of Keystone trusts.

This post will cover details of the recently implemented instance-users blueprint, which makes use of keystone domains to contain users related to credentials which are deployed inside instances created by heat.  If you just want to know how the new stuff works, you can skip to the last sections :)

So...why does heat create users at all?

Lets start with a bit of context.  Heat has historically needed to do some or all of the following:
  1. Provide metadata to agents inside instances, which poll for changes and apply the configuration expressed in the metadata to the instance.
  2. Signal completion of some action, typically configuration of software on a VM after it is booted (because nova moves the state of a VM to "Active" as soon as it spawns it, not when heat has fully configured it)
  3. Provide application level status or metrics from inside the instance, e.g to allow AutoScaling actions to be performed in response to some measure of performance or quality of service.
Heat provides API's which enable all of these things, but all of those API's require some sort of authentication, e.g credentials so whatever agent is running on the instance is able to access it.  So credentials must be deployed inside the instance, e.g here's how things work if you're using the heat-cfntools agents:

heat-cfntools agents data-flow with CFN-compatible API's