The Puppet Anchor Pattern in Practice

Recently, my team rewrote all Puppet manifests for Mayflower’s core infrastructure with our two year long Puppet experience in mind. Many mistakes were made in the past, but this time we decided to write clean and structured Puppet code based on the latest language features from Puppet 3. More blog posts about advanced Puppet topics are going to be published in 2014.

The main theme of the rewrite was abstraction. Our goal was to use as many third-party modules as possible. Because many modules needed fixes, we contributed a lot of code back to the community. We adopted the Roles, Profiles, Components Design and use Hiera extensively to separate business logic, modules, manifests and data. Our manifests basically glue the modules together and embed them sensibly in our environment, seeded with data from Hiera.

The Problem

If you’ve written your fair share of Puppet manifests, you can imagine that such an architecture requires the definition and inclusion of lots of classes. Sadly, Puppet’s dependency system treats classes different than resources: If you include class foo in class bar, then class foo and all its contained resources and dependencies won’t have any dependency relations with class bar.

Consider the following example that demonstrates this in Puppet code:

class bar {
  include foo
  notify { 'bar': }
}

class foo {
  notify { 'foo': }
}

include bar

If you include foo in bar, you can’t be sure which notify will be realized first.

The following dependency graph explains the dependency situation. Dashed arrows (in both directions) are resource containment dependencies: Resources defined in classes („contained in the class“) are realized with the class, not before or after it. Solid arrows denote explicitly defined dependencies.

Trying require

If you’ve read the Puppet documentation you might know that there is a special version of include called require. Quoting the Puppet Language Guide:

The require function acts like include, but also causes the class to become a dependency of the surrounding container.

So we modify class bar accordingly:

class bar {
  require foo
  notify { 'bar': }
}

class foo {
  notify { 'foo': }
}

include bar

Which yields the following dependency graph:

As you can see, the order of the classes and their contained notify resources is now correct as expected. This solution, however, isn’t universal. Puppet will ensure that foo is realized before bar, but doesn’t care if other resources are realized in between.

But sometimes this behavior is not desirable. In that case foo and bar should be wrapped into a self-contained „module“, which is a class that ensures its child classes are realized like contained resources in classes.

Chaining it up

To illustrate wrapping, two new classes were introduced into the example code. The first one, end, must be realized after foo  and bar . The second class is called wrapper and should contain all other classes in the right order.

Our first try looks like this:

class bar {
  notify { 'bar': }
}

class foo {
  notify { 'foo': }
}

class end {
  notify { 'end': }
}

class wrapper {
  class { 'foo': } ->
  class { 'bar': } ->
  class { 'end': }
}

include wrapper

Please note the use of chaining arrows. The semantics of class { ‚foo‘: }  are the same as those of include foo but the former version allows using chaining arrows.

As you would’ve surely expected, this version doesn’t wrap correctly because we’re basically using include in class wrapper.

However, note that we would have needed two requires in bar and end for foo and bar if we wouldn’t have used the much nicer chaining arrow syntax. Also note that we now define dependencies in the parent of foo, bar and end, namely wrapper, and not inside the individual classes. This allows us to immediately recognize and understand the dependency structure of the wrapped classes without looking at them one by one.

Wrapping it up

So we still need a classic containment dependency relationship like that of contained resources in classes because we want the wrapped classes to be realized with the wrapper class. In current Puppet versions, this isn’t possible without a hack.

This hack is called the Anchor Pattern.

The trick is to use a resource called anchor that does nothing but act as a marker for dependencies. If you define an anchor in a class, it will be contained inside the class like a resource as usual. If you add a regular before or -> dependency to a class, this class will be realized after the anchor. So far so good. Now you just need to add a second anchor and another dependency between the class and new anchor like this:

class wrapper {.
  anchor { 'wrapper::begin': } ->
  class { 'foo': }             ->
  class { 'bar': }             ->
  class { 'end': }             ->
  anchor { 'wrapper::end': }
}

As both anchors are contained in the wrapper class and there is a end-to-end dependency chain from the begin to the end anchor with dependent classes in between, those classes (and their contained resources) will be realized after the begin and before the end anchor.

Therefore, these classes are contained inside the wrapper class like regular resources using both anchor resources.

New Puppet language feature: contain

Of course the Anchor pattern is not ideal. It’s both unnecessary code bloat and at lot more resources to manage in the catalog and the dependency graph which slows down puppet even more. If you have a complex setup, this will also confuse you while debugging dependency cycles.

A Puppet bug about the class containment situation already exists and was fortunately fixed recently: #8040.

With Puppet 3.4 the new contain function will be introduced which is similar to include and require. It includes a class and contains it just like a resource. Yay!

The wrapper class can now be written like this:

class wrapper {
  contain foo
  contain bar
  contain end

  Class['foo'] ->
  Class['bar'] ->
  Class['end']
}

Of course you could also use require in the classes bar and end for foo and bar respectively instead of the chaining arrow syntax.

Conclusion

You should now know how to use and depend on classes and their resources correctly using Anchors and the new contain function. You should use them where needed, but especially in modules.

Sadly, we encounter lots of dependency fuckups in third-party modules because many module authors don’t care about the dependencies of their modules. A second Puppet run should never be an option! Please fix your modules and your manifests!