Mastering Ansible

In this course, we will implement a 3-tier web application topology using one (1) NGINX load-balancer, two (2) application nodes running Apache2 and Python, one (1) MySQL database server, and one (1) control machine for running Ansible. If you want to follow along, you can setup an environment that fits these requirements:

1. Access to a control machine that can run Ansible
2. Servers running Ubuntu Trusty (14.04) - 5 in total
3. SSH keypair trusts from the control machine to all other nodes (so no password is required to login via ssh)
4. Access via a user that has superuser (sudo) privileges on the end hosts

In the Appendix (Lecture 55), I provide details on the docker setup that I use throughout the course.

In the materials below, is a Vagrantfile that can be used to create a similar vagrant setup. Rename "Vagrantfile.txt" to "Vagrantfile" and read the "README" comments inside of the file to get started.

The first step in using Ansible is installing Ansible! In this lecture, we will:

• Install the Ansible tools using a package manager (apt-get for the Ubuntu-based demo environment)
• Test the installed binaries by checking the version on the command line

Ansible needs to know all of the hosts that it can connect to and manage. In this lecture, we will cover:

• The concept of the inventory file and how Ansible uses it
• The default Ansible inventory file format and location (/etc/ansible/hosts).

We need to populate an inventory file so that Ansible can reach our environment hosts. Rather than populate the default location on the control machine, we can create a simple file in our local repository and tell Ansible to use it, by default. In this lecture, we will:

• Configure a local inventory file with hostnames and groups (dev)
• Configure a local config file to use the new inventory file by default (ansible.cfg)

Once the inventory is populated, you can tell Ansible to execute against all or some subset of the included hosts. In this lecture, we'll:

• Use host patterns to target a subset of the full inventory: all, *, <hostname>, <groupname>, :, !

It's time to execute Ansible against our environment hosts. In this lecture, we will:

• Learn about the basic building block of Ansible: the task.
• Understand the structure of tasks: modules and arguments.
• Execute ad-hoc tasks using the ansible command with the ping and command modules.

Running ad-hoc Ansible commands are great for troubleshooting and discovery, but the real power of Ansible is in coding those tasks into playbooks that you can save and run, as needed. In this lecture, we will:

• Group tasks for a common set of hosts together into plays.
• Create our first playbook (hostname.yml) to encode the command task into an Ansible YAML file.

With our playbook written, it's time to execute it and review the output. In this lecture, we will:

• Execute the hostname.yml playbook against hosts using the ansible-playbook command.
• Review the output of the ansible-playbook command.

Now that you understand the framework for encoding tasks and executing playbooks, we will start building up our application environment. For every task, we'll find a relevant module and add the necessary parameters to our playbooks. You can browse the module documentation if you're looking for the right module or parameters to fit your needs.

We start our configuration with installing the necessary packages for each tier. In this lecture, we will:

• Install packages with the apt module
• Set package installed state using one of: present, latest, absent.
• Ensure the apt cache is updated before package installation with update_cache=yes

While executing Ansible tasks, you may run into "permission denied" errors. This generally means the task you're executing requires some sort of superuser or sudo privilege. In this lecture, we will:

• Enable privilege escalation in Ansible with become/sudo at the play level. This will resolve "permission denied" errors on the end hosts where superuser privileges are required.

• Execute the configuration playbooks to apply the package installation changes on the load-balancer and database hosts.

For the webserver hosts, we need to install multiple packages via apt. Rather than create four individual apt tasks, we'll learn about a new Ansible feature: loops using with_items. In this lecture, we will:

• Loop through a task multiple times by providing a list of inputs using with_items
• Install multiple packages with a single task on the webserver hosts using with_items

With the packages installed, we want to get a handle on the services we'll need to get up and keep running. In this lecture, we will:

• Manage running service state with the service module
• Enable services to start on system startup with enabled=yes

• Write a support playbook to do a full stack restart using the service module

• Enable apache2 modules using the apache2_module ansible module.
• Configure handlers to restart services only when needed.
• Trigger handlers after configuration changes using notify.

• Copy files and directories from a local machine to a target host
• Set file attributes on the target host such as owner, group, and file mode

• Install python dependencies using the pip module and a requirements.txt file
• Setup a python virtual environment to host the python web application

• Create and remove symbolic links using the file module.

• Inject variables into files before copying to the target host with the template module.
• Implement for loops in a jinja2 template.

• Edit a single line in a file on the target host using regular expressions and the lineinfile module.

• Create a mysql database using the mysql_db module.
• Create a mysql user and set the password and allowed host using the mysql_user module.

• Write a support playbook to check the status of the application stack.
• Check that a host is listening on a specific TCP port and interface using the wait_for module.

• Extend the stack status playbook to perform http requests against the application using the uri module.
• Save the contents of the HTTP request using the register module
• Check the contents of a variable and conditionally cause a failure using the when and fail modules.

• Review the progress made on configuring the demo application environment.

• Understand the purpose and structure of roles in ansible.
• Use the ansible-galaxy command to create a role skeleton.

• Convert an ansible playbook to a role by copying tasks and handlers.
• Address ordering issues by moving service tasks after configuration tasks.

• Convert an ansible playbook to a role by copying files and templates.
• Address ordering issues by moving service tasks after configuration tasks.

• Group multiple playbooks into a single playbook using the include module.

• Use system facts as variables to configure host-specific values.
• Query a target host for system facts using the setup module.

• Provide default values for variables defined in a role.

• Understand variable precedence in ansible.
• Define variable values using vars.
• Define variable values when including roles to override role defaults.

• Use a dictionary/hash and with_dict to execute a task multiple times.

• Execute a shell command on a target host with the shell module.
• Capture the output of a shell command with the register module.
• Loop through the items in a registered array variable using with_items.
• Conditionally perform an action on the target host using when.

• Convert the remaining playbooks to roles.
• Convert a file to a template to parameterize values and leverage passed variables.

• Define variables in a separate file using group_vars

• Encrypt variable values using ansible-vault.
• Configure a vault password file to store the vault passphrase.
• Add a configuration default to use a vault password file automatically.

• Learn about the ansible public roles repository: galaxy.ansible.com.
• Understand tradeoffs and considerations of using public roles.

Now that our application is configured, and we've completed the bulk of our refactoring into roles, we will look at ways to optimize our playbooks and speed up our execution time.

• Disable fact gathering to speed up execution using gather_facts.

• Increase apt cache timer with the apt module and cache_valid_time.
• Extract a repeated sequential task and perform it in parallel across all hosts to reduce execution time.

• Limit playbook execution to a subset of hosts without modifying plays

• Add user-defined tags to tasks.
• Limit playbook execution by including (--tags) or excluding (--skip-tags) defined tags.

• Override ansible's built-in logic to define when a task is marked as changed.
• Use a python function (.keys()) in an ansible when clause to determine the task will be marked as changed.

• Learn about ansible transport optimizations to reduce execution time and overhead.
• Learn about when to use accelerated mode and ssh pipelining.

• Learn how to resolve ordering problems that cause playbook execution dead-ends
• Skip problematic tasks using ignore_errors.

• Step through tasks one-by-one using --step.
• Display a list of tasks included in a playbook using --list-tasks.
• Jump to a task and begin execution at an arbitrary point using --start-at-task.

• Use the ansible provided retry host list to re-run a playbook on failed or unreachable hosts.

• Validate the syntax of an ansible playbook using --syntax-check.
• Perform a dry-run (no-op) of an execution against a live host using --check

• Print a msg or the contents of a variable using the debug module.

• Understand SSH public/private key architecture.
• Learn how to create an RSA keypair.
• Learn how to setup an SSH trust to login to a host without using a password.

• Understand the basics of the YAML format: files, keys/values, hashes, and lists.

• Understand the basics of the Jinja2 templating language: variables, templates, and loops

• Setup a development environment to test ansible using docker-machine and docker-compose.