IT Infrastructure Automation with Puppet - A Complete Guide!
3.3 (13 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
124 students enrolled

IT Infrastructure Automation with Puppet - A Complete Guide!

No more manual work! Automate your IT infrastructure with ease with the latest techniques in Puppet!
3.3 (13 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
124 students enrolled
Created by Packt Publishing
Last updated 10/2018
English [Auto]
Current price: $139.99 Original price: $199.99 Discount: 30% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
  • 12.5 hours on-demand video
  • 1 downloadable resource
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Build complex configuration files with dynamic data using Puppet's EPP template mechanism.
  • Manage files, directories, links, users, and packages with Puppet.
  • Learn to manage dependencies and execute commands on remote servers.
  • Secure Puppet deployments and master methods to harden infrastructure using community modules.
  • Create a scalable and performance-efficient Puppet infrastructure.
  • Configure a highly available Puppet infrastructure along with storeconfigs and external node classifiers.
  • Understand configuration using Docker and Puppet along with orchestration using Mcollective.
  • Basic understanding of Puppet Configuration Management will be useful (Not mandatory).

With the ever-increasing challenges of IT infrastructures, it's no surprise that IT professionals are looking for ways to automate as much as possible! The need to automate common activities has become the major requirements in many large organizations. Puppet is one such configuration management tool that allows to efficiently build, manage and automate all your IT infrastructure efficiently! If you’re a developer, a system administrator, or you're simply curious about learning Puppet, then go for this Learning Path!

This comprehensive 4-in-1 course is a comprehensive guide, filled with quick step-by-step instructions that will help you automate your IT infrastructure with the latest techniques in Puppet! Initially, you’ll get a run-through of the strategies and patterns to introduce Puppet automation. Write custom plugins for Puppet such as facts, resource types, providers, and functions. Also, Deploy configuration files and templates at superfast speeds and manage user accounts and access control appear like a cakewalk. Finally, automate your IT infrastructure with Puppet.

By the end of the course, you’ll become a pro with Puppet 5 by automating your IT infrastructure with the latest techniques!

This Learning Path takes a practical approach with step-by-step recipes to conduct effective penetration testing using the powerful Kali Linux! At the very beginning, you’ll develop your skills using attacks such as wireless cracking, Man-in-the-Middle, and Denial of Service (DOS). Also, explore tools such as Metasploit, Wireshark, Nmap, and much more to detect vulnerabilities with ease! Finally, you’ll master all phases in a typical penetration testing project focusing on relevant Windows tools and techniques.

By the end of the course, you’ll be one step ahead of hackers by discovering and patching your network vulnerabilities as well as performing professional-level web penetration testing!

Contents and Overview

This training program includes 4 complete courses, carefully chosen to give you the most comprehensive training possible.

The first course, Puppet 5 – From Beginner to Pro, is a practical guide with the latest features of Puppet 5. This course teaches you how to install packages and config files, create users, set up scheduled jobs, provision cloud instances, build containers, and so much more. Every example in this course deals with something practical that you're likely to need in your work, and you'll see the complete Puppet code that makes it happen, along with step-by-step instructions on what to type and what output you'll see.

The second course, Puppet Configuration Management, covers Puppet to automate all your work! In this course, you will be introduced to Puppet development and learn how to set up your own private development environment, so that you can develop your code without fear of hurting live environments. From there on out we get familiar with the Puppet DSL and then deploy our code to a puppet server and puppet node like you would in a production environment. After completing this class you should have all the handles you need to start utilizing puppet and know how to further progress and become a true Puppeteer!

The third course, Puppet Techniques for Enterprise Deployments, covers Automating your IT infrastructure with the latest techniques in Puppet! This video course will begin by covering methods of reporting with Puppet and configuring various types of open source dashboards along with procedures to use report processors and create custom ones. You will then exploit PuppetDB to configure and create efficient Puppet reports and also to monitor Puppet changes and alert based on the same. Finally, you will learn about how to implement Nagios based monitoring using Puppet. Towards the end of the video course, you will concentrate on Puppet security. You will also learn about procedures to secure the Puppet deployments and methods to harden infrastructure using community modules. Finally, we’ll learn how to debug errors in the Puppet environment.

The fourth course, Advanced Puppet Techniques, covers writing custom plugins for Puppet such as facts, resource types, providers, and functions. This video course starts with taking your Puppet knowledge to the next level by delving deep into concepts such as scaling the Puppet infrastructure, utilizing storeconfigs with PuppetDB, and using different external node classifiers. You’ll then get your hands dirty by running Puppet components in Docker containers, using Puppet to create Docker images and running applications on Docker along with orchestration using Mcollective. At the end of the course, you will learn how to write custom plugins for Puppet such as facts, resource types, providers, and functions.

By the end of the course, you’ll be able to automate your IT infrastructure with ease with the latest techniques in Puppet!

About the Authors

  • John Arundel is a DevOps consultant, which means he helps people build world-class web operations teams and infrastructures and has fun doing it. He was formerly a senior operations engineer at global telco Verizon, designing resilient, and high-performance infrastructures for major corporations such as Ford, McDonald's, and Bank of America. He is now an independent consultant, working closely with selected clients to deliver web-scale performance and enterprise-grade resilience on a startup budget. He likes writing books, especially about Puppet (Puppet 2.7 Cookbook and Puppet 3 Cookbook are available from Packt). He also provides training and coaching on Puppet and DevOps, which, it turns out, is far harder than simply doing the work himself. Off the clock, he is a medal-winning, competitive rifle and pistol shooter and a decidedly uncompetitive piano player. He lives in a small cottage in Cornwall, England and believes, like Cicero, that, if you have a garden and a library, then you have everything you need. You may like to follow him on Twitter at @bitfield.

  • Toni Schmidbauer has been working with various Unix/Linux and BSD systems for the last 17 years. He started working with configuration management in 2005 with CFEngine 2 and switched to Puppet in 2010. Currently, he is responsible for a Puppet infrastructure containing around 2000 nodes running Solaris, AIX, and Linux. When he's not writing Puppet code he likes to tinker with different programming languages and computers in general.

  • Anirban Saha is an infrastructure professional with more than seven and half years' experience in infrastructure management at various industries and organizations ranging from early startups to corporate environments. He has worked extensively with configuration management and automation tools including Puppet, Chef, Ansible, Saltstack, and Terraform, to name a few. He has extensive experience in architecting, deploying, and managing large infrastructures and speaks at various conferences on the latest technologies.

Who this course is for:
  • System administrators, DevOps professionals, and Developers who want to master advanced topics in Puppet as well as implement added functionalities that Puppet provides for enterprise-level deployments.
Course content
Expand all 88 lectures 12:27:51
+ Puppet 5 – From Beginner to Pro
35 lectures 03:13:46

This video gives an overview of the entire course.

Preview 03:00

What do we mean by configuration data? There will be lots of pieces of information in your manifests which we can regard as configuration data. Let’ see them in detail.

  • Take a look at why data needs to be maintained

  • See the settings that depend on nodes and operating systems that differ

  • Set up Hiera

Why Hiera?

Let’ see how to use Hiera to store and query configuration Data.

  • Troubleshoot Hiera

  • Query Hiera

Adding Hiera Data to Your Puppet repo

In this video, we'll look at how to write Hiera data sources, including string, array, and hash data Structures.

  • Take a look at Single values, Boolean values

  • Learn about Arrays and Hashes

Types of Hiera Data

Now, we’ll see how to interpolate values into Hiera strings using lookup(), including Puppet facts and other Hiera data, and how to duplicate Hiera data structures using alias().

  • Use lookup()

  • Use alias()

  • Use literal()

Interpolation in Hiera Data

So far, we've only used a single Hiera data source. Actually, you can have as many data sources as you like. Each usually corresponds to a YAML file, and they are listed in the hierarchy section of the hiera.yaml file.

  • Deal with multiple values

  • See merge behaviors

  • Take a look at Data sources based on facts

The Hierarchy

We've seen how to use Hiera data to fill in the title and attributes of resources in the manifest. We can now take this idea one step further and create resources directly from Hiera queries.

  • Build resources from Hiera arrays

  • Build resources from Hiera hashes

Creating Resources with Hiera Data

Puppet often needs to know your secrets; for example, passwords, private keys, and other credentials need to be configured on the node, and Puppet must have access to this information. Let’s see how to do that.

  • Set up GnuPG

  • Add an encrypted Hiera source

  • Create and edit an encrypted secret

Managing Secret Data

Although you could write your own manifests for everything you want to manage, you can save yourself a lot of time and effort by using public Puppet modules wherever possible.

  • See Puppet Forge

  • Use the r10k module management tool

Using Puppet Forge Modules

Now that we know how to find and install public Puppet modules, let's see how to use them.

  • Use puppetlabs/mysql

  • Use puppetlabs/apache

  • Use puppet/archive

Using Modules in Your Manifests

One of the oldest-established Puppet Forge modules is puppetlabs/stdlib, the official Puppet standard library. Let's look more closely now and see what the standard library provides and where you might use it.

  • Safely installing packages with ensure_packages

  • Modify files in place with file_line

  • See some other useful functions

Exploring the Standard Library

As we've seen, a Puppet module is a way of grouping together a set of related code and resources that performs some particular task, like managing the Apache web server or dealing with archive files. But how do you actually create a module?

  • Create a repo and write module code

  • Create and validate the module metadata

  • Install and apply your module

Writing Your Own Modules

We've come across the class concept a few times so far, without really explaining it. Let's explore a little further now and see how to use this key Puppet language building block.

  • Learn about the class keyword

  • Declare parameters to classes

  • See parameter data types


Whereas a class lets you group together related resources, a defined resource type lets you create new kinds of resources and declare as many instances of them as you like.

  • Look at defined resource type definition

  • Define type aliases

Defined Resource Types

Let’s see how to use the classes array in Hiera to include common classes on all nodes, and other classes only on particular nodes.

  • Use include with lookup()

  • Take a look at common and per-node classes

Managing Classes with Hiera

Now that we know how to include different sets of classes on a given node, depending on the job the node is supposed to do, let's think more about how to name those classes in the most helpful way.

  • Create a class called role::app_server

  • Rewrite the app_server role to include profiles

Roles and Profiles

The methods we’ve learnt so far are very useful, but they are limited in one respect: they can only use static text, rather than building the contents of the file dynamically. Let’s see how we can do that with templates.

  • See the dynamic data problem

  • Take a look at Puppet template syntax

What Are Templates?

Since the end result of a template is a file, you won't be surprised that we use Puppet's file resource to work with templates. In fact, we use an attribute of the file resource that you've seen before: the content attribute.

  • Reference template files

  • See computations in templates

  • See conditional statements in templates

Using Templates in Your Manifests

If we can generate parts of a file from Puppet expressions, and also include or exclude parts of the file depending on conditions, could we generate parts of the file with a Puppet loop? 

That is to say, could we iterate over an array or hash, generating template content for each element?

  • Iterate over Facter data and structured facts

  • Iterate over Hiera data and work with templates

  • Pass parameters to templates and render templates on the command line

Iteration in Templates

Although the technology behind containers is at least thirty years old, it's only in the last few years that containers have really taken off. This is largely thanks to the rise of Docker, a software platform which makes it easier to create and manage containers.

  • Take a look at the deployment problem

  • Build Docker containers

Understanding Containers and Deployment with Docker

Puppet can certainly install and manage the Docker service for you, just as it can any other software, but it can also do a lot more.

  • Install Docker

  • Run multiple instances of container

Managing Docker with Puppet

Of course, it's very useful to be able to download and run public images from Docker Hub or other registries, but to unlock the real power of Docker we need to be able to build and manage our own images too.

  • Build images from Dockerfiles

  • Manage Dockerfiles

Managing Docker Images

Although Dockerfiles are a fairly powerful and flexible way of building containers, they are only static text files, and very often you will need to pass information into the container to tell it what to do. We might call such containers—whose configuration is flexible and based on data available at build time—dynamic containers.

  • Configure containers

Building Dynamic Containers

There are two ways of persisting data in a container: the first is to mount a directory from the host machine inside the container, known as a host-mounted volume, and the second is to use what's called a Docker volume. We'll look at both of these in this video.

  • See Host-mounted volumes

  • Take a look at Docker volumes

Persistent Storage for Containers

We started off the section by saying that containers are completely self-contained, and have no access to each other, even if they're running on the same host. But to run real applications, we need containers to communicate. Fortunately, there is a way to do this: the Docker network.

  • Learn container orchestration

Networking and Orchestration

If a container can contain a whole operating system, such as Ubuntu, you might be wondering: "can't I just run Puppet inside the container?" Let’s find out.

  • Configure containers with Puppet

Running Puppet Inside Containers

Before exploring the advantages of cloud computing, perhaps we should define what it is. In the pre-cloud days, if you needed computing power, you bought an actual, physical computer. 

But from the customer's point of view, we don't necessarily want a computer: we just want to compute.

  • Automate cloud provisioning

  • Use CloudFormation and Terraform

Introducing the Cloud

Let’s see how to set up a new account and get the credentials you need to start building infrastructure with Puppet.

  • Create an AWS account

  • Create IAM policy and user

  • Store AWS credentials

Setting Up an Amazon AWS Account

In this video, we'll see how to generate an SSH key pair to connect to your EC2 instances, and also install the puppetlabs/aws module with its dependencies.

  • Create a key pair

  • Install the puppetlabs/aws module

  • Installing the AWS SDK gem

Getting Ready to Use puppetlabs/aws

Although you can manage many different types of AWS resources with Puppet, the most important is the EC2 instance. In this video, we'll see how to create your first EC2 instance.

  • Choose an Amazon Machine Image (AMI)

  • Create the EC2 instance

  • Access your EC2 instance

Creating EC2 Instances with Puppet

In the previous video, we used the pre-existing default VPC and subnet to create our instance. In production you'll want to use a dedicated VPC for your Puppet-managed resources, to keep it separate from any other resources. You could, for example, have a staging VPC and a production VPC.

  • Create an instance in a custom VPC

  • See other AWS resource types

Managing Custom VPCs and Subnets

There's nothing wrong with managing AWS resources directly in the code, as we've done previously, but we can do just a little bit better. Let’s see how.

  • Iterate over Hiera data to create resources

  • Clean up unused resources

Provisioning AWS Resources from Hiera Data

It's now time to see how all the ideas from the previous sections fit together. It should be helpful for you to see how a complete Puppet infrastructure works, and you can also use this repo as a basis for your own projects.

  • Copy the repo

  • Understand the demo repo

Getting the Demo repo

You will need to change some of the data and settings in the demo repo to be able to use it yourself. Let’s see how to do that.

  • Configure users

  • Add per-node data files and role classes

  • Modify the bootstrap credentials

Adapting the repo for Your Own Use

If you'd like to try out the demo repo on a Vagrant box, there is a suitable Vagrantfile included within the repo directory.

  • Bootstrap a Vagrant VM

  • Bootstrap physical or cloud nodes

  • Use other distributions and providers

Bootstrapping a New Node
Test Your Knowledge
6 questions
+ Puppet Configuration Management
19 lectures 03:50:42

This video will give an overview of entire course

Preview 03:11

In this video, we will learn how to install a puppet server on a CentOS 7 machine. We will also install the puppet agent on a second virtual machine. We will do our first puppet run.

  • Bootstrap the course environment

  • Install the puppet server in VM1

  • In this video, we will learn how to install a puppet server on a CentOS 7 machine. We will also install the puppet agent on a      second virtual machine. We will do our first puppet run

Installing Puppet Server

In this video we will learn about different puppet configuration files relevant for the puppet master and the agent.

  • Identify the most important configuration files.

  • Locate the important configuration files.

  • Gain insights into the import configuration settings.

Puppet Configuration Files

In this video, we will learn how certificates work in a puppet client/server architecture. We will also see how the puppet server controls which client is able to access the server using certificates.

  • Explain why and how puppet uses certificates

  • Explain how a puppet agent requests a certificate

  • Gain insights into import configuration settings

Understanding Certificates

In this video, we are going to learn how certificate signing and revoking work. We will also cover cleaning up of stale client certificates.

  • Locate certificates

  • Working with certificate signing requests

  • Cleaning up and regenerating agent certificates

Working with Certificates

In this video, we are going to explain how certificate auto signing works. We will also learn about different ways to control auto signing of agent certificates.

  • Explain what auto signing is used for

  • Identify the different types of auto signing

  • Identify important configuration settings

Configuring Auto Signing

We will look at Facter, a tool to gather data about systems. Facter is used by Puppet for accessing system information.

  • Explain what Facter is and why it is required for Puppet.

  • Using Facter to collect system information.

  • Extending Facter with Custom facts.

Understanding Facter

We will look under the hood of a puppet run. We will also describe running the puppet agent on a regular basis via cron or as a daemon.

  • Explain the steps required for a successful puppet run.

  • Explain Puppet catalog.

  • Run Puppet from cron or as an agent.

Going Over a Puppet Run

We will learn how puppet uses the main manifest to compile a catalog for a puppet node. We will also see how puppet finds additional puppet modules and get a short introduction to puppet environments.

  • Explain how puppet finds the main manifest

  • Explain how puppet finds additional modules

  • A short introduction to puppet environments

The Main Manifest(s)

We will learn how to assign specific configurations to specific nodes. We will also learn how to specify node definitions covering more than one node.

  • Explain node definitions

  • Explain the different forms of node definitions

  • Using an external node classifier

Node Definitions

We will explain what puppet environments are and how we can assign different environments to nodes. We will also learn about environment specific settings.

  • Explain puppet environments

  • Explain how environments are assigned to nodes.

  • Understand how to set environment-specific settings.

Understanding Puppet Environments

We will learn about version control systems and their importance in managing puppet code on production servers. We are going introduce GIT, a distributed version control system.

  • Explain version control.

  • Introduce GIT, a distributed version control system.

  • Give examples of repository hosting.

Understanding Version Control

We are going to explain basic git usage patterns.

  • Creating a repository.

  • Making commits.

  • Merging and working with the log

Creating a Central Repository

We will set up a central repository for storing our puppet code and demonstrate basic usage. We will also mention possibilities for running a central git server.

  • Creating a central repository.

  • Pushing/Pulling commits.

Basic GIT Usage

We will learn how to integrate GIT with r10k, a tool for deploying puppet code. This covers using GIT branches for creating puppet environments. We will also deploy a custom module with the help of r10k.

  • Introduce r10k, a tool for puppet code deployments.

  • Understand the control repository.

  • Deploy custom modules with r10k.

Deploying Puppet Code with r10k

We will learn how to use the roles and profiles pattern to organize our puppet manifests.

  • Explain the theory behind the roles and profiles pattern.

  • Explain a practical example.

  • Configure an example role and profile.

The Roles and Profiles Pattern

We will learn how separating data and code allows more flexibility when writing puppet code. Hiera, a hierarchical database allows us to separate data and code in Puppet.

  • Explain the purpose of hiera, a hierarchical database.

  • Explain hiera data lookup.

  • Use hiera for assigning classes to nodes.

Separating Data and Code with Hiera

We will learn how to run puppet in debug or noop mode to review changes. We will also create a graphical representation of the resource graph.

  • Identify the important log files.

  • Run the puppet agent in debug mode.

  • Displaying the resource graph.

Troubleshooting the Puppet Agent

We will learn where to find import log files and how to run the puppet server in the foreground and debug mode. We will also demonstrate some tools for debugging connection problems.

  • Identify important log files.

  • Run the puppet master in the foreground and debug mode.

  • Demonstrate the tools for debugging connection problems.

Troubleshooting the Puppet Master
+ Puppet Techniques for Enterprise Deployments
16 lectures 02:06:14

This video will give an overview of the entire course

Preview 04:23

Configuration and management of infrastructure is important. Equally important is the process of reporting, without which there would be no way to find out if the infrastructure is behaving the way we want it to.

  • Understand Puppet reports and configuration parameters for reporting

  • Explore built-in report processors

  • Configure and test reporting in Puppet server and agents

Understanding and Configuring Reporting in Puppet Server and Agent

PuppetDB already provides us with enormous possibilities to find out extremely granular details of our infrastructure. Puppetboard, a community project, leverages the features of PuppetDB to provide a very decent dashboard for reporting.

  • Understand and install Puppetboard as Puppet dashboard

  • Create the Python config files and configure Apache web server

  • Run and explore the Puppetboard dashboard

Puppetboard as a Puppet dashboard

Puppetexplorer is another community project to assist users in reporting and displaying an attractive dashboard to use PuppetDB data.

  • Understand and explore Puppetexplorer as Puppet dashboard

  • Install and configure Puppetexplorer

  • Explore Puppetexplorer dashboard to view Puppet report data

Puppetexplorer as a Puppet Dashboard

Dashboards are definitely useful for viewing reports. However, their usefulness increases many folds as soon as we are able to manipulate data on the dashboard and integrate it with tools such as Puppet to manage the infrastructure. Foreman is one such tool.

  • Understand, install and configure Foreman

  • Configure report processor for Foreman

  • Explore Foreman dashboard and view reports

Configuring Foreman as a Puppet Dashboard

Apart from acting as a reporting dashboard for Puppet, Foreman has additional features such as acting as an External Node Classifier for Puppet. With minimum changes to the configuration, an efficient ENC can be configured on the Foreman.

  • Understand and configure foreman-proxy on the Puppet Server

  • Configure Puppet proxy and authentication rules for Foreman proxy

  • Configure ENC script to provide catalog and push facts

Using Foreman as an External Node Classifier

The community has huge support of all kinds and the number of resources that can be found to be used readily are overwhelming. However, at times, the need to build custom solutions is unavoidable. Puppet has support for custom reporting which can be used to create reporting mechanisms of various types.

  • Understand report processor and directory structure

  • Create report processor and add code to send email reports

  • Demonstrate email report generation via Puppet run

Understanding and Creating Custom Report Processors

Puppetboard and Puppetexplorer are definitely great resources for readily implementing Puppet reporting. However, Puppet is flexible enough to allow us to create our own reports from PuppetDB by querying the data.

  • Understand query mechanism of PuppetDB

  • Create Ruby script to query PuppetDB and fetch data

  • Use PuppetDB data to create formatted file and HTML file

Creating and Displaying Puppet Reports from PuppetDB

One of the excellent dashboard platform out in the infrastructure space is the Elastic stack created with Elasticsearch, Logstash, and Kibana. It has taken data collection and dashboards to a whole new level and Puppet can also take advantage of it by sending reports to Logstash and viewing data in Kibana.

  • Understand the Elastic stack applications and tools

  • Configure the Elastic stack using Puppet modules

  • Use Puppet module to configure custom reporter for Logstash

Puppet Reporting Using Elasticsearch, Logstash, and Kibana

One of the oldest monitoring tool in the infrastructure space, Nagios has its exclusive Puppet types available. When used with PuppetDB, it can create a dynamic and efficient monitoring workflow.

  • Explore available Nagios Puppet types

  • Create Nagios module with server and client manifests

  • Use PuppetDB to export and collect virtual resources for dynamic configurations

Implementing Nagios Monitoring Using Puppet

Using Puppet to deploy changes is great but making sure that Puppet is running when it is supposed to run is one of the critical tasks to keep in mind. Using a custom script and the power of Nagios, this task can be make very simple and elegant.

  • Understand the concept of Puppet run time monitoring

  • Create Nagios monitoring script to monitor Puppet run time

  • Integrate script with Nagios infrastructure and run checks successfully

Monitoring Puppet Agent Run Using Nagios

In a fairly large infrastructure, making sure that only valid hosts are able to get catalogs from the Puppet server is of utmost importance. With policy based certificate signing feature of Puppet, this additional security step is possible.

  • Understand policy based certificate signing

  • Create a script to check certificate signing request on Puppet server

  • Add certificate signing request attributes to agent nodes

Using Policy Based Certificate Signing for Puppet Nodes

Complex workflows often require that some configurations or data are kept restricted for special nodes or resources. Restricted mount points can be configured in Puppet to exactly implement this requirement by modification of few configuration files.

  • Understand and create new Puppet mount point

  • Add restriction to mount point by editing auth.conf file

  • Test restricted mount point on agent nodes

Restricting Access to Puppet Mount Points

Tracking active changes or finding what will change without actually changing them is often a requirement in highly critical infrastructures and can also be used for audit purposes. Puppet metaparameters are extremely helpful in implementing such workflows.

  • Understand audit and noop metaparameters

  • Test the metaparameters in a module

  • Track changes using noop metaparameter and resources type

Tracking Changes Using Puppet Metaparameters

Identifying vulnerable system components and then keeping them secure is one of the base requirements of any infrastructure. The enormous database of community modules provide us with a lot of modules which are specifically created to help with this process.

  • Identify system components where security is applicable

  • Explore community modules for security and hardening

  • Test security modules on agents

Exploring Community Modules for System Hardening

No tool or platform can be deployed successfully without running into errors and problems. It can be tedious to find out underlying problems from error messages. However, some of the most common problems and errors can be readily identified with some help.

  • Follow best practices to avoid errors and problems

  • Identify problems from the errors

  • Apply solution for the identified problem

Troubleshooting Puppet Server and Client Errors
Test Your Knowledge
5 questions
+ Advanced Puppet Techniques
18 lectures 03:17:09

This video provides an overview of the entire course. 

Preview 05:54

As infrastructure grows to hundreds and thousands of nodes, the capacity of the Puppet server reaches a limit. Load balancing in multiple Puppet servers is an optimal way to handle load and provide redundancy.

  • Understand a highly available and scalable Puppet architecture

  • Configure Puppet servers to work with load balancers

  • Configure Apache HTTP server as a load balancer for Puppet service

Load Balancing in Puppet Servers to Handle Scaling

As infrastructure grows, isolating the Puppet server components is a better way to manage them, rather than having a monolithic Puppet server node. Let’s take a look at isolating and centralizing the Puppet certificate authority service, which is an important step in this process.

  • Understand the split CA architecture

  • Create and configure the Puppet CA node

  • Integrate the Puppet CA node with the load balanced infrastructure

Creating a Centralized CA Server for Authentication

With the growth in the number of agent nodes, maintaining a database and the history of node configuration and metadata is crucial for any infrastructure. PuppetDB helps to maintain this database and also aids in configuration via exported resources.

  • Install and configure the PostgreSQL database

  • Install and configure the PuppetDB, and connect it to the PostgreSQL database

  • Integrate PuppetDB with Puppet server

Understanding Exported Resources and Configuring PuppetDB

Infrastructure management involves services with server and client based models. The server has to push or pull the data from various nodes in the infrastructure for which it needs the node specific information. To automate this process, the methods of exporting and collecting in Puppet modules are extremely useful.

  • Define virtual or exported resources in Puppet client modules

  • Send exported resources information to PuppetDB

  • Get exported resources data from PuppetDB and generate configuration using server modules

Exporting and Collecting in Modules using PuppetDB

After the data from thousands of nodes are stored in a database, they are not very useful, unless they can be queried and visualized using structured querying mechanisms. Different Puppet query mechanisms and dashboards helps in solving these problems.

  • Configure workstation, and understand methods to query PuppetDB

  • Understand PuppetDB API endpoints and perform queries.

  • Use default PuppetDB dashboard for visualization

Using the PuppetDB API and Dashboard

As the number of nodes and service roles grow in an infrastructure, it is important to have proper node classification mechanisms so that nodes are able to receive catalogs correctly and without latency. Proper manageable node classification methods are also desirable.

  • Understand external node classification concept

  • Create scripts to provide node definition and classification data

  • Configure Puppet server to use scripts as source of node classification data

External Node Classification Using Scripts

Hostname based node classification often gets very complex and it is easier to switch to other node classification methods. Apart from providing data isolation from code, Hiera can also be an effective framework to provide node classification data based on metadata such as facts.

  • Understand Hiera file structure for node classification

  • Configure Hiera for node classification

  • Populate Hiera data file with fact specific data and module information

Node Classification Using Hiera

Running Puppet components in Docker containers is a bit different than doing it on a physical or virtual machine. It is important to understand how the service components work in an isolated manner and also in the form of an application stack.

  • Understand containerized service deployment method and configure Docker on workstation

  • Configure and run Puppet server in Docker container.

  • Configure and run Puppet server side components using Docker Compose

Running Puppet server side components in Docker Containers

Running Puppet agent in Docker Containers to automate them is different from the general process; given the fact that container based dynamic environments don’t have hostnames for the containers and system based process control systems are unavailable in containers.

  • Create a Docker file for Puppet agent

  • Run Docker containers with Puppet agent

  • Use Hiera to fetch service specific catalog data

Automating Docker Containers Using Puppet Agent

The fact that containers are able to be spawned within few seconds is true. This is due to the fact that Docker images already have all the configurations baked in them and provisioning during the container launch is not required, thus bringing down the deployment time. Creating images is a crucial step in container based environments.

  • Build and push image manually

  • Build images with dependencies using image_build Puppet module

  • Use Packer to build, tag and push Docker images

Creating Docker Images

Once Puppet server side components and containers are ready to be spawned using Docker images, it is important that we are able to launch application stacks involving multiple components such as application and database. Docker Compose can be managed efficiently by use of respective Puppet modules and help us in managing container based application stacks.

  • Understand community modules to manage Docker

  • Create Puppet module to configure Docker environment

  • Create Puppet module to deploy application stacks using Docker Compose

Running Applications in Docker Using Puppet

Once infrastructure reaches a considerable size, a requirement is to be able to perform specific tasks on specific nodes. Managing and maintaining an inventory of role based nodes is a tedious process. MCollective can be a very efficient tool in implementing orchestration which helps us to target nodes based on metadata and run specific tasks

  • Understand Orchestration and various components of the framework

  • Configure ActiveMQ message broker as the middleware

  • Configure MCollective servers and clients using Puppet modules

Configuring MCollective for Orchestration in Puppet

Although facter provides a lot of system facts, often in complex environments, they are not enough. By writing custom facts based on our infrastructure, we can add meaningful custom data to our infrastructure nodes based on which complex tasks can be performed.

  • Understand custom fact concept and directory structure

  • Write custom fact

  • Make facts dependent on system or other custom facts

Writing Custom Facts

Puppet built-in types are very helpful in writing configurations. However, for simplified use of Puppet modules, often writing custom types are very important. For users who don’t wish to get into the complex methods of writing Ruby based Puppet types, resource types are a simpler alternative where custom types can be written using the Puppet language.

  • Learn concept of resource type

  • Write custom resource type using ‘define’ method

  • Apply resource type to agent nodes

Writing Custom Resource Types

With the numerous tools and services available in the infrastructure ecosystem, it is quite exhausting and time consuming to write Puppet modules for them from scratch using built-in Puppet types. For such scenarios, advanced Puppet types can be written using Ruby which can simplify custom tasks.

  • Understand custom Puppet type concept and directory structure

  • Create custom type and understand options

  • Apply custom Puppet type to agent nodes

Writing Custom Puppet Types

Puppet types are useful in defining custom tasks and processes, however without Providers, Puppet types are unable to do much. If Puppet types are the interface to create custom configurations, providers are the backend logic which actually performs the tasks on the systems. Multiple providers can be written for systems or services with different behaviors.

  • Understand custom provider concept and directory structure

  • Create custom provider and add functionality based on Puppet type

  • Apply custom provider to agent nodes

Writing Custom Providers

Although Puppet types and Providers help us in creating advanced tasks and interfaces, for some smaller tasks and scenarios, it is enough to create functions. Custom functions can be created in the Puppet language and Ruby to handle such scenarios.

  • Understand custom function concept and directory structure

  • Create custom function and use it in Puppet manifest

  • Apply manifest to agent nodes

Writing Custom Functions
Test Your Knowledge
5 questions