Sensu - Intermediate

This lecture will involve using the official Sensu Puppet module to install components, configure checks, etc.

We'll start with a base Ubuntu Trusty vagrant image, install Puppet, Sensu, RabbitMQ, and Redis. The installation of each component will be 100% reproducible, and the Puppet code I use will be on Github.

In this lecture we will continue using Puppet to install Uchiwa, the Dashboard. We'll also fine-tune RabbitMQ vhost/user/passwords.

We'll also configure Sensu to use those same credentials, so they can both talk to each other.

We conclude with a discussion on how to integrate Sensu Puppet modules with the rest of your Puppet infrastructure.

What would it look like if you could install an application and deploy the monitoring for that application in a single Puppet class? We'll use the example with an Apache webserver and check_http.

This lecture will involve using the official Sensu Chef cookbook to install server and client components and get them up and running from scratch. We'll use chef-solo to run things, and knife to install the cookbooks we need.

While we are at it we'll install Uchiwa too, and make sure everything is pretty in the Dashboard. All the chef code we write will be on Github of course.

In this second half of using Chef with Sensu, we cover adding checks and handlers. We'll install a plugin to check disk space, and an email handler so you can get email when you run out of disk space!

This lecture will involve using the Unofficial ansible playbook from Matthieu Maury (Mayeu on Github) to deploy Sensu using Ansible.

We'll use the Ansible-Galexy tool to get the playbooks we need, including Redis, RabbitMQ, and Uchiwa.

Finally we'll look at the dashboard and make sure everything is up and running. Of course all the Ansible code we write will be available on Github.

Next we take the Ansible playbooks and integrate email handler, so you can get emails when things go wrong.

Then we'll install a disk check to watch how much disk space your servers have.

Finally I'll show how to combine Sensu and Apache together, so your monitoring gets deployed with your app.

This lecture will involve using the official Sensu Salt forumulas to install Sensu, RabbitMQ, Redis, and Uchiwa.

We'll connect all the pieces, put our config data in Pillar, and verify everything works on Uchiwa. Of course all the Salt formulas we right will be available on Github.

Next we'll use Salt to install a disk check, and an email handler so you can get emails when your disks fill up.

Finally, we conclude by demonstrating it what it would look like if we deployed Sensu checks with apache, to get good cohesion between your deployments and your monitoring.

How do you know what commands to run, on what hosts, how frequently, and with what settings?

This will cover things like:

• Interval
• Occurences
• Refresh
• Command
• Subscribers

Some of these paramaters may not be as obvious as you might think.

We covered one example of how to take advantage of the third-party Sensu Community Plugins, but let's do more.

Specifically we'll look at exactly what it takes to install any of the plethora of Sensu Plugins: http://sensu-plugins.io/plugins/

Luckily, writing your own Sensu checks is Super Easy!

This lecture will cover how to do that.

Also, if you decide to write a check in Ruby, you can take advantage of the included 'sensu-plugin/check/cli' class to get a lot of free boilerplate.

We'll start with a simple bash script, and grow it into a fancy ruby script, all in a test-driven-development way. Fancy!

Writing Sensu handlers is a bit more complicated than writing checks. This lecture will cover how to write basic handlers and how to test them.

I'll bring out my Android phone and write a notify-my-android handler together.

Sensu comes packaged in "omnibus" form, which means every platform Sensu is deployed to will get a consistent, modern ruby that you can use to run checks.

Learn how to take advantage of this ruby, install custom gems, etc.

Standalone checks, Safemode, and check Subscriptions are intermediate Sensu topics that are often confused.

Let's get this all figured out together on the whiteboard.

We've talked about handlers a bit, but this lecture will go deeper into the nut-and-bolts on how they work, and how they interact with filters.

Filters are essential for tuning noisy alerts that don't deserve to wake people up in the middle of the night, and every production Sensu installation should use them.

Any production Sensu installation needs to be secured with SSL. This prevents the possibility of attackers ease-dropping and listening on the plain-text communication.

It also allows RabbitMQ to validate the client connections, to ensure no rouge actors can connect.

This does require a bit more work and understanding, but you have an expert to help show you the way...

Running any production system requires extra security considerations. In this lecture I demonstrate possible attack surfaces of Sensu, and how to harden them against attack.

This includes where you should be using encryption, authentication, and network separation.

I'll cover specifics about what to do with RabbitMQ, Redis, Uchiwa, and Sensu in a secure, production environment.

The sensu-cli is a great tool for interacting with Sensu. In this lecture I'll demonstrate how to install, configure, and use the sensu-cli.

Then I'll start getting all crazy mixing in 'jq' and 'xargs'. If you love stringing unixy tools together with pipes to construct powerful command line invocations, you will love this lecture.

Sending events directly to the Sensu socket on localhost is probably the most under-rated feature of Sensu.

This will be a small introduction to this feature, but the possibilities are endless.

This is the idea that you can iterate over some dynamic endpoint, and emit your own check results based on that status of the endpoint.

I'll start with a simple bash example: a nightly cron job. Then I'll grow it into a complicated custom report, iterating through customers and sending Sensu notification on a per-customer basis.

This course should leave you with enough knowledge and inspiration to run Sensu in a Production environment.

It should be secure, and sustainably deployed, and as an engineer you should feel comfortable and empowered by Sensu.