
This video gives an overview of the entire course.
What do we mean by configuration data? There will be lots of pieces of information in your manifests which we can regard as configuration data. Let’ see them in detail.
• Take a look at why data needs to be maintained
• See the settings that depend on nodes and operating systems that differ
• Set up Hiera
Let’ see how to use Hiera to store and query configuration Data.
• Troubleshoot Hiera
• Query Hiera
In this video, we'll look at how to write Hiera data sources, including string, array, and hash data Structures.
• Take a look at Single values, Boolean values
• Learn about Arrays and Hashes
Now, we’ll see how to interpolate values into Hiera strings using lookup(), including Puppet facts and other Hiera data, and how to duplicate Hiera data structures using alias().
• Use lookup()
• Use alias()
• Use literal()
So far, we've only used a single Hiera data source. Actually, you can have as many data sources as you like. Each usually corresponds to a YAML file, and they are listed in the hierarchy section of the hiera.yaml file.
• Deal with multiple values
• See merge behaviors
• Take a look at Data sources based on facts
We've seen how to use Hiera data to fill in the title and attributes of resources in the manifest. We can now take this idea one step further and create resources directly from Hiera queries.
• Build resources from Hiera arrays
• Build resources from Hiera hashes
Puppet often needs to know your secrets; for example, passwords, private keys, and other credentials need to be configured on the node, and Puppet must have access to this information. Let’s see how to do that.
• Set up GnuPG
• Add an encrypted Hiera source
• Create and edit an encrypted secret
Although you could write your own manifests for everything you want to manage, you can save yourself a lot of time and effort by using public Puppet modules wherever possible.
• See Puppet Forge
• Use the r10k module management tool
Now that we know how to find and install public Puppet modules, let's see how to use them.
• Use puppetlabs/mysql
• Use puppetlabs/apache
• Use puppet/archive
One of the oldest-established Puppet Forge modules is puppetlabs/stdlib, the official Puppet standard library. Let's look more closely now and see what the standard library provides and where you might use it.
• Safely installing packages with ensure_packages
• Modify files in place with file_line
• See some other useful functions
As we've seen, a Puppet module is a way of grouping together a set of related code and resources that performs some particular task, like managing the Apache web server or dealing with archive files. But how do you actually create a module?
• Create a repo and write module code
• Create and validate the module metadata
• Install and apply your module
We've come across the class concept a few times so far, without really explaining it. Let's explore a little further now and see how to use this key Puppet language building block.
• Learn about the class keyword
• Declare parameters to classes
• See parameter data types
Whereas a class lets you group together related resources, a defined resource type lets you create new kinds of resources and declare as many instances of them as you like.
• Look at defined resource type definition
• Define type aliases
Let’s see how to use the classes array in Hiera to include common classes on all nodes, and other classes only on particular nodes.
• Use include with lookup()
• Take a look at common and per-node classes
Now that we know how to include different sets of classes on a given node, depending on the job the node is supposed to do, let's think more about how to name those classes in the most helpful way.
• Create a class called role::app_server
• Rewrite the app_server role to include profiles
The methods we’ve learnt so far are very useful, but they are limited in one respect: they can only use static text, rather than building the contents of the file dynamically. Let’s see how we can do that with templates.
• See the dynamic data problem
• Take a look at Puppet template syntax
Since the end result of a template is a file, you won't be surprised that we use Puppet's file resource to work with templates. In fact, we use an attribute of the file resource that you've seen before: the content attribute.
• Reference template files
• See computations in templates
• See conditional statements in templates
If we can generate parts of a file from Puppet expressions, and also include or exclude parts of the file depending on conditions, could we generate parts of the file with a Puppet loop?
• That is to say, could we iterate over an array or hash, generating template content for each element?
• Iterate over Facter data and structured facts
• Iterate over Hiera data and work with templates
• Pass parameters to templates and render templates on the command line
Although the technology behind containers is at least thirty years old, it's only in the last few years that containers have really taken off. This is largely thanks to the rise of Docker, a software platform which makes it easier to create and manage containers.
• Take a look at the deployment problem
• Build Docker containers
Puppet can certainly install and manage the Docker service for you, just as it can any other software, but it can also do a lot more.
• Install Docker
• Run multiple instances of container
Of course, it's very useful to be able to download and run public images from Docker Hub or other registries, but to unlock the real power of Docker we need to be able to build and manage our own images too.
• Build images from Dockerfiles
• Manage Dockerfiles
Although Dockerfiles are a fairly powerful and flexible way of building containers, they are only static text files, and very often you will need to pass information into the container to tell it what to do. We might call such containers—whose configuration is flexible and based on data available at build time—dynamic containers.
• Configure containers
There are two ways of persisting data in a container: the first is to mount a directory from the host machine inside the container, known as a host-mounted volume, and the second is to use what's called a Docker volume. We'll look at both of these in this video.
• See Host-mounted volumes
• Take a look at Docker volumes
We started off the section by saying that containers are completely self-contained, and have no access to each other, even if they're running on the same host. But to run real applications, we need containers to communicate. Fortunately, there is a way to do this: the Docker network.
• Learn container orchestration
If a container can contain a whole operating system, such as Ubuntu, you might be wondering: "can't I just run Puppet inside the container?" Let’s find out.
• Configure containers with Puppet
Before exploring the advantages of cloud computing, perhaps we should define what it is. In the pre-cloud days, if you needed computing power, you bought an actual, physical computer.
• But from the customer's point of view, we don't necessarily want a computer: we just want to compute.
• Automate cloud provisioning
• Use CloudFormation and Terraform
Let’s see how to set up a new account and get the credentials you need to start building infrastructure with Puppet.
• Create an AWS account
• Create IAM policy and user
• Store AWS credentials
In this video, we'll see how to generate an SSH key pair to connect to your EC2 instances, and also install the puppetlabs/aws module with its dependencies.
• Create a key pair
• Install the puppetlabs/aws module
• Installing the AWS SDK gem
Although you can manage many different types of AWS resources with Puppet, the most important is the EC2 instance. In this video, we'll see how to create your first EC2 instance.
• Choose an Amazon Machine Image (AMI)
• Create the EC2 instance
• Access your EC2 instance
In the previous video, we used the pre-existing default VPC and subnet to create our instance. In production you'll want to use a dedicated VPC for your Puppet-managed resources, to keep it separate from any other resources. You could, for example, have a staging VPC and a production VPC.
• Create an instance in a custom VPC
• See other AWS resource types
There's nothing wrong with managing AWS resources directly in the code, as we've done previously, but we can do just a little bit better. Let’s see how.
• Iterate over Hiera data to create resources
• Clean up unused resources
It's now time to see how all the ideas from the previous sections fit together. It should be helpful for you to see how a complete Puppet infrastructure works, and you can also use this repo as a basis for your own projects.
• Copy the repo
• Understand the demo repo
You will need to change some of the data and settings in the demo repo to be able to use it yourself. Let’s see how to do that.
• Configure users
• Add per-node data files and role classes
• Modify the bootstrap credentials
If you'd like to try out the demo repo on a Vagrant box, there is a suitable Vagrantfile included within the repo directory.
• Bootstrap a Vagrant VM
• Bootstrap physical or cloud nodes
• Use other distributions and providers
This video provides an overview of the entire course.
The aim of the video is to gain familiarity with puppet core concepts and its architecture.
Learn why do we need configuration management tools
Learn puppets architecture
Learn puppet resource decleration
The aim of this video is to show how to create virtual machines for puppet master and nodes.
Download and Install the Virtualization Client
Download Images for the operating system
Create virtual machines using the virtualization client
The aim of this video is to show how to do a basic puppet master installation.
Set the IP Address and the hostname
Set the hostname
Install NTP and set the timezone
The aim of this video is to show how to configure puppet master for a basic setup.
Install the packages
Make the production directory environment
Configure puppet master using the configuration file puppet.conf
Security is a large issue in automation tools. This video aims to guide on how to configure security in Puppet.
Configure SELinux
Configure firewall to allow Puppet connections
The aim of this video is to show how to install the Puppet node.
Configure Network and Hostname
Install NTP and set time zone
Install the puppet agent packages
The aim of this video is to show different configuration options on the puppet master and the puppet node.
Learn different configurations of Puppet.conf on the puppet master
Learn different configurations of Puppet.conf on the puppet node
Learn how to edit puppet.conf via the command line
The aim of the video is to gain familiarity about the resource abstraction layer and then the imperative style of programming used in puppet.
Learn what is Resource Abstraction Layer
Learn how catalogs are compiled for a puppet Node.
Learn puppet resource declaration
The aim of this video is to show you how to create user Manifests.
Node definition for Puppet Nodes
Define the User Manifest
Run the puppet Agent
The aim of this video is to show you how to create File Manifests.
Define the File Manifests
Learn how to edit files
Run the puppet Agent
The aim of this video is to show you how to create Package Manifests.
Define the Package Manifest
Run the puppet Agent
The aim of this video is to show you how to create Service Manifests.
Define the Service Manifest
Run the puppet Agent
The aim of the video is to learn how to sign certificates from the puppet master and also learn how to automatically sign certificates for a particular domain.
Learn how to sign Puppet Node certificates on the puppet Master
Learn how to automatically sign certificates for a particular domain
The aim of this video is to learn in detail about executing a Puppet run against the Puppet Master.
Learn how to run the puppet Agent
Learn the flags which effect the puppet Agent Run
Learn how to execute a Puppet Agent Dry Run
The aim of this video is to learn how to add additional nodes to an existing Puppet Setup.
Install a new Puppet Node
Configure the new Puppet Node
Integrate the puppet node with the Puppet Master
The aim of this video is to learn how to how to regenerate and revoke certificates from the Puppet master.
Revoke a Certificate
Regenerate the certificate
The aim of this video is to learn how to do a proper node definition using site.pp.
Learn default Node definitions
Learn multiple node Definitions
Learn how to define nodes using regular expressions
The aim of the video is to learn how to use selectors and how to create puppet classes.
Learn how to use Selector Variables
Learn how to create classes
Learn how to call these classes in the Node definition file
The aim of this video is to learn how to use and create puppet modules.
Learn how to create Modules
Learn Module structure
Learn how to call a module from the node definition file
The aim of this video is to learn how to use puppet namespaces and auto-loading for modularity.
Learn Puppet Namespaces
Learn Auto-loading
Learn how to use puppet namespaces and Auto-loading in Puppet Modules
The aim of this video is to learn how to use Puppet Variables in Modules.
Create a Variable class file
Use variables as parameters to classes
Learn how to reference variables using their name
The aim of this video is to learn how to create configuration files using Puppet File Templates.
Create a Configuration Class
Learn how to use the inline_template function
Learn how to populate dynamic content using ERB
The aim of this video is to learn how to create external puppet facts.
Create an external facts directory
Define the external fact in the file
Verify its working using the facter command
The aim of the video is to learn how to configure Hiera to get configuration data into puppet.
Modify main puppet configuration file to include Hiera
Use Hiera YAML files to define hierarchy
Restart puppet service
The aim of this video is to learn how to use Hiera in Modules.
Introduce Hiera Variables in Module
Run the puppet agent
The aim of this video is to learn how to write those files from which Hiera does its lookups and where the key value pairs are stored.
Configure directory where Hiera YAML files are stored
Create YAML file to store Hiera variables
Run puppet agent on Puppet node
The aim of this video is to learn how to use Hiera to invoke classes for our puppet nodes.
Modify node definition file
Use Hiera include function
Run puppet agent on Puppet node
The aim of the video is learn how to install puppet agent on Windows.
Setup Windows Virtual Machine
Download Puppet Agent
Run the Installer
The aim of this video is to learn how to generate the agent certificates from the puppet agent installed on our Windows machine.
Run the Puppet Agent On Windows Virtual Machine
Verify stored certificates
The aim of this video is to learn how to use puppet namespaces and auto-loading for modularity.
Install puppet module to download files of internet
Make windows node definition and class
Run puppet agent
The aim of this video is to learn how to manage files and packages on the windows puppet node with puppet.
Modify windows node definition
Include service type resource declaration
Run the puppet agent
The aim of this video is to learn the concepts of puppet roles and profiles.
Learn concepts of Puppet Roles
Learn concepts of Puppet Profiles
The aim of the video is to learn how to setup a Github Account.
Go to the GitHub Site
Create GitHub Account
Add SSH Key in Github Account
The aim of this video is to learn how to add files to a Git repository.
Configure Git in Puppet Master
Create a GitHub local Repository
Make a commit to the local repository
The aim of this video is to learn how to make and apply changes to our repository and push those changes to our master repository.
Add location of remote Github repository
Push the commits made in local repository to remote repository
The aim of this video is to learn how to get back to a working configuration and code if something breaks.
Make a wrong Commit
Push to local and remote Repository
Recover with initial commit
This video provides an overview of the entire course.
Maintaining a complete system with multiple servers and applications is challenging, but Puppet can make it easier by automatically keeping things up to date.
Describe Puppet
Explore a system automated with Puppet
Demonstrate using Puppet to automatically install software
To automate our systems, we need a reliable tool that is continually monitoring our system state.
Understand the importance of idempotence for reliability
See how Puppet continuously monitors a system
Understand the Puppet design approach and how to apply Puppet
To set up Puppet, we need to understand its architecture and how to configure it.
See where Puppet stores configuration information
See how to get a Puppet server running
Understand how to separate Puppet code into multiple environments
Puppet agents connect to the server over HTTP/S using certificate authentication, which needs to be configured correctly.
Configure an agent to connect to the Puppet server
Understand certificates and certificate authorities
Demonstrate how to sign a certificate on the server for a new Puppet agent
Puppet has its own language for specifying the desired system state. Understanding how resources work will help us understand Puppet code.
See the layout of Puppet code files in a Puppet environment
Understand how Puppet resources specify the desired system state
Install a package using a Puppet resource definition
To control multiple systems with Puppet, we need a way to apply resources only to selected systems.
Organize a Puppet resource in a node definition
Use a regular expression to select multiple Puppet nodes
Review the parts of a resource definition
Puppet works to guarantee that we create a consistent, reliable configuration for a particular system.
See how Puppet builds a catalog of all the resources applied to a node
Apply a Puppet node catalog to a node
Observe how Puppet protects us from creating an inconsistent catalog
To get started with Puppet, we need to at least know how to use it to install packages, configure files, and start services.
Use Puppet to install and update packages
Use Puppet to create and configure a file
Use Puppet to enable and start a service
If we want Puppet to reliably process our resource definitions in the right order, we need to specify resource dependencies.
Understand why specifying dependencies is good practice
Specify a dependency using the “require” metaparameter
Specify a dependency using the “before” metaparameter
When a configuration changes, we need Puppet to respond correctly by running commands or restarting services.
See the distinction between applying a resource and refreshing a resource
Cause a resource to be refreshed using the “subscribe” metaparameter
Cause a resource to be refreshed using the “notify” metaparameter
Puppet allows us to manage multiple types of systems from the same Puppet server using environments.
Run Puppet in “agent” mode against a server
Configure a Puppet environment
Configure an agent to belong to a specific Puppet environment
We want to apply Puppet code to multiple nodes without having to copy/paste resources into multiple node definitions.
Organize resources into a Puppet class
Create a module and declare a class in the module’s manifests
Include a class in a Puppet node manifest
To make our classes more reusable, we want to provide parameters from the outside that our class can use.
Declare a parameter in a Puppet class
Provide a default value for a parameter
Make a parameter optional or mandatory
Puppet strictly controls duplicate class declarations, so we need to understand how to declare classes correctly.
Include classes in node manifests
Declare a class together with its parameters
Observe how includes and declarations work together to avoid duplicate resources
A Puppet module includes not only classes, but also files and other items needed to apply the classes.
Organize files in a Puppet module to deliver them to a remote system
Use Puppet’s file server to send files to an agent
Configure Puppet to manage an entire directory tree of configuration files
Larger modules need to organize Puppet code into multiple classes or into defines to better manage how Puppet resources are applied.
Split a module into multiple classes to use it more flexibly
Create a define to apply similar sets of resources multiple times
Declare resources in a define in a way that avoids duplicate resources
We need to apply different configurations to different nodes based on the features of that node, so we need a tool to collect facts about a node.
Collect node facts with Facter
Use Facter to query for node facts
Use facts from Facter in Puppet code
We need some piece of information from a node that is not already collected by Facter, so we need a custom fact.
Place a Ruby code file in the correct location in a Puppet module
Create a new fact that collects data from a node
Allow Puppet to automatically deploy the fact, and then use it
We need the ability to apply configuration to groups of nodes without duplication and with the ability to modify it easily.
Configure Hiera to read data from YAML files
Write YAML files to provide configuration to nodes
Update Puppet code to look up configuration data from Hiera
Using Hiera to feed parameters to classes is so common. Puppet can do it automatically for us as long as we use the right naming convention.
Create a Hiera configuration item to automatically provide a parameter to a class
Understand how priority works for Puppet class parameters
Simplify our Puppet code to remove explicit class parameter values
Using Hiera with more complex data structures allows us to radically simplify our Puppet manifests.
Understand the lookup function and Hiera merge strategies
See an example of deep merging of hash data in Hiera
Use a Hiera array merge to remove duplication and simplify our Puppet node manifests
To properly configure the software we install, we need to manage configuration files using parameters from our classes.
Understand the purpose of templates
Create a template file using embedded Puppet
Generate a configuration file from a template
The Embedded Puppet language allows us to control how our templates are created.
Use both printing and non-printing tags
Control whitespace to improve the quality of the template output
Allow our templates to access class parameters
To ensure that our templates are used correctly, we need the ability to declare and enforce parameters.
Review the difference between class scope and template scope
Declare mandatory and optional parameters in a template
Pass a parameter to a template from Puppet code
More advanced templates need more than just printing variables. We need to be able to select and repeat blocks of content.
Use conditionals to select blocks of content in a template
Use iteration to repeat blocks of content in a template
Use data transformation functions to put data in the right format for the configuration file
The Puppet server provides catalog information to nodes, but it does not store history on what was applied. To store and search that history, we need PuppetDB.
Understand how PuppetDB works with Puppet server
Install and configure PuppetDB
Configure Puppet server to report to PuppetDB
To retrieve data from PuppetDB, we need to learn the Puppet query language.
Use the Puppet query language to query for resources
Select what fields are returned from a query
Use the Puppet query language to query for facts
We can run Puppet queries using the command line, HTTP requests with cURL, or our Puppet code.
Use the Puppet query command to run a query
Use cURL to run a query over HTTP
Query from Puppet code and use the result in a catalog
To access complex information from queries, we need to know how to combine a parent and a subquery to get to the data we want.
See an example of querying for node data
Build a subquery that queries from resources but returns data from nodes
Run the complex query and see the results
If we use PuppetDB with Puppet, we can use exported resources declared on one node to manage configuration on other nodes.
Understand the purpose of exported resources
Export an SSH key resource from one node
Collect the SSH key resource on a different node and add the host key to SSH
Video Learning Path Overview
A Learning Path is a specially tailored course that brings together two or more different topics that lead you to achieve an end goal. Much thought goes into the selection of the assets for a Learning Path, and this is done through a complete understanding of the requirements to achieve a goal.
It's no surprise that the need to automate common activities has become the major requirement in many large organizations. Automation provides faster deployments and fewer failures, improves security, and reduces costs for your organization. Puppet is an ideal tool for automating IT infrastructure and Configuration Management in your organization and this course will show you how.
In this well thought out Learning Path, you'll go from beginner to confident Puppet user with a series of clear, practical examples to help you manage every aspect of your server setup. You’ll learn to implement consistent configuration across every server and deploy applications to multiple servers.
Moving further, you’ll get hands-on with managing your IT infrastructure the Puppet 5 platform. You’ll build a custom software installation and configuration to match your application infrastructure. Explore the latest features of Puppet 5 by executing, testing, and deploying Puppet across your systems in no time.
By the end of this Course, you’ll be able to control your IT infrastructure effectively and become a Configuration Management master with the power and flexibility of Puppet 5.0.
Key Features
Develop the skills to run Puppet 5, the latest version of this stable and mature technology, without hiccups.
Get easy solutions to ensure the correct server configuration (Windows or Linux server).
Use Puppet with hands-on examples to build a reliable & secure automated infrastructure and create your own Puppet code.
Author Bios
John Arundel is a DevOps consultant, which means he helps people build world-class web operations teams and infrastructures and has fun doing it. He was formerly a senior operations engineer at global telco Verizon, designing resilient, high-performance infrastructures for major corporations such as Ford, McDonald's, and Bank of America. He is now an independent consultant, working closely with selected clients to deliver web-scale performance and enterprise-grade resilience on a startup budget. He likes writing books, especially about Puppet (Puppet 2.7 Cookbook and Puppet 3 Cookbook are available from Packt). He also provides training and coaching on Puppet and DevOps, which, it turns out, is far harder than simply doing the work himself. Off the clock, he is a medal-winning, competitive rifle and pistol shooter and a decidedly uncompetitive piano player. He lives in a small cottage in Cornwall, England and believes, like Cicero, that, if you have a garden and a library, then you have everything you need. You may like to follow him on Twitter at @bitfield.
Abdul Basit Alvi is a certified professional with expertise in open source cloud systems and mobile core networks. He has over 3 years of experience in deploying and managing open source cloud systems and using puppet for configuration management and deployment of large scale cloud systems such as OpenStack. He also has experience in using configuration management tools such as Ansible and Puppet for Orchestration and Automation of various virtual network functions in mobile core networks.
Alan Hohn is a Software Architect who primarily works in Java, Go, and Python. He has a lot of experience re-architecting embedded systems and in combining embedded and enterprise approaches. Currently, he works as a Lockheed Martin Fellow encouraging the adoption of virtualization, DevOps, and cloud technologies. This also means that he frequently tries out new technologies and new approaches. He enjoys teaching on technical topics and enjoys hearing from those who find his courses useful. He is employed by Lockheed Martin Rotary and Mission Systems and his personal blog is called Variegated.