Learning Path: Puppet: The One-Stop Guide to Puppet

This video will give an overview of the entire course.

Configuration and management of infrastructure is important. Equally important is the process of reporting, without which there would be no way to find out if the infrastructure is behaving the way we want it to.

PuppetDB already provides us with enormous possibilities to find out extremely granular details of our infrastructure. Puppetboard, a community project, leverages the features of PuppetDB to provide a very decent dashboard for reporting.

Puppetexplorer is another community project to assist users in reporting and displaying an attractive dashboard to use PuppetDB data.  

Dashboards are definitely useful for viewing reports. However, their usefulness increases many folds as soon as we are able to manipulate data on the dashboard and integrate it with tools such as Puppet to manage the infrastructure. Foreman is one such tool.

Apart from acting as a reporting dashboard for Puppet, Foreman has additional features such as acting as an External Node Classifier for Puppet. With minimum changes to the configuration, an efficient ENC can be configured on the Foreman.

The community has huge support of all kinds and the number of resources that can be found to be used readily are overwhelming. However, at times, the need to build custom solutions is unavoidable. Puppet has support for custom reporting which can be used to create reporting mechanisms of various types.

Puppetboard and Puppetexplorer are definitely great resources for readily implementing Puppet reporting. However, Puppet is flexible enough to allow us to create our own reports from PuppetDB by querying the data.

One of the excellent dashboard platform out in the infrastructure space is the Elastic stack created with Elasticsearch, Logstash, and Kibana. It has taken data collection and dashboards to a whole new level and Puppet can also take advantage of it by sending reports to Logstash and viewing data in Kibana.

One of the oldest monitoring tool in the infrastructure space, Nagios has its exclusive Puppet types available. When used with PuppetDB, it can create a dynamic and efficient monitoring workflow.

Using Puppet to deploy changes is great but making sure that Puppet is running when it is supposed to run is one of the critical tasks to keep in mind. Using a custom script and the power of Nagios, this task can be make very simple and elegant.

In a fairly large infrastructure, making sure that only valid hosts are able to get catalogs from the Puppet server is of utmost importance. With policy based certificate signing feature of Puppet, this additional security step is possible.

Complex workflows often require that some configurations or data are kept restricted for special nodes or resources. Restricted mount points can be configured in Puppet to exactly implement this requirement by modification of few configuration files.

Tracking active changes or finding what will change without actually changing them is often a requirement in highly critical infrastructures and can also be used for audit purposes. Puppet metaparameters are extremely helpful in implementing such workflows.

Identifying vulnerable system components and then keeping them secure is one of the base requirements of any infrastructure. The enormous database of community modules provide us with a lot of modules which are specifically created to help with this process.

No tool or platform can be deployed successfully without running into errors and problems. It can be tedious to find out underlying problems from error messages. However, some of the most common problems and errors can be readily identified with some help.

This video provides an overview of the entire course.

As infrastructure grows to hundreds and thousands of nodes, the capacity of the Puppet server reaches a limit. Load balancing in multiple Puppet servers is an optimal way to handle load and provide redundancy. 

As infrastructure grows, isolating the Puppet server components is a better way to manage them, rather than having a monolithic Puppet server node. Let’s take a look at isolating and centralizing the Puppet certificate authority service, which is an important step in this process.

With the growth in the number of agent nodes, maintaining a database and the history of node configuration and metadata is crucial for any infrastructure. PuppetDB helps to maintain this database and also aids in configuration via exported resources.

Infrastructure management involves services with server and client based models. The server has to push or pull the data from various nodes in the infrastructure for which it needs the node specific information. To automate this process, the methods of exporting and collecting in Puppet modules are extremely useful.

After the data from thousands of nodes are stored in a database, they are not very useful, unless they can be queried and visualized using structured querying mechanisms. Different Puppet query mechanisms and dashboards helps in solving these problems.

As the number of nodes and service roles grow in an infrastructure, it is important to have proper node classification mechanisms so that nodes are able to receive catalogs correctly and without latency. Proper manageable node classification methods are also desirable.

Hostname based node classification often gets very complex and it is easier to switch to other node classification methods. Apart from providing data isolation from code, Hiera can also be an effective framework to provide node classification data based on metadata such as facts.

Running Puppet components in Docker containers is a bit different than doing it on a physical or virtual machine. It is important to understand how the service components work in an isolated manner and also in the form of an application stack.

Running Puppet agent in Docker Containers to automate them is different from the general process; given the fact that container based dynamic environments don’t have hostnames for the containers and system based process control systems are unavailable in containers.

The fact that containers are able to be spawned within few seconds is true. This is due to the fact that Docker images already have all the configurations baked in them and provisioning during the container launch is not required, thus bringing down the deployment time. Creating images is a crucial step in container based environments.

Once Puppet server side components and containers are ready to be spawned using Docker images, it is important that we are able to launch application stacks involving multiple components such as application and database. Docker Compose can be managed efficiently by use of respective Puppet modules and help us in managing container based application stacks.

Once infrastructure reaches a considerable size, a requirement is to be able to perform specific tasks on specific nodes. Managing and maintaining an inventory of role based nodes is a tedious process. MCollective can be a very efficient tool in implementing orchestration which helps us to target nodes based on metadata and run specific tasks

Although facter provides a lot of system facts, often in complex environments, they are not enough. By writing custom facts based on our infrastructure, we can add meaningful custom data to our infrastructure nodes based on which complex tasks can be performed.

Puppet built-in types are very helpful in writing configurations. However, for simplified use of Puppet modules, often writing custom types are very important. For users who don’t wish to get into the complex methods of writing Ruby based Puppet types, resource types are a simpler alternative where custom types can be written using the Puppet language.

With the numerous tools and services available in the infrastructure ecosystem, it is quite exhausting and time consuming to write Puppet modules for them from scratch using built-in Puppet types. For such scenarios, advanced Puppet types can be written using Ruby which can simplify custom tasks.

Puppet types are useful in defining custom tasks and processes, however without Providers, Puppet types are unable to do much. If Puppet types are the interface to create custom configurations, providers are the backend logic which actually performs the tasks on the systems. Multiple providers can be written for systems or services with different behaviors.

Although Puppet types and Providers help us in creating advanced tasks and interfaces, for some smaller tasks and scenarios, it is enough to create functions. Custom functions can be created in the Puppet language and Ruby to handle such scenarios.