Mastering Python Networking

This lesson uses practical examples to illustrate how to use Python to execute commands on a network device. It will also discuss the challenges of having a CLI-only interface in automation. The chapter will use the Pexpect and Paramiko libraries for the examples.

While it is hard to imagine any networking device coming out today without an Application Program Interface (API) for programmatic communication, it is a known fact that many of the network devices deployed in previous years did not contain API interfaces. The intended method of management for those devices was through Command Line Interfaces (CLIs) using terminal programs, which were originally developed with a human engineer in mind. The management relied on the engineer's interpretation of the data returned from the device for appropriate action. As the number of network devices and the complexity of the network grew, it became increasingly difficult to manually manage them one by one. Let’s look at it in detail.

Before we dive into the packages, let's examine the options of putting together a lab for the benefit of learning.

Similar to the original Tcl Expect module by Don Libe, Pexpect launches or spawns another process and watches over it in order to control the interaction. The Expect tool was originally developed to automate interactive processes such as FTP, Telnet, and rlogin, and was later expanded to include network automation. Unlike the original Expect, Pexpect is entirely written in Python, which does not require TCL or C extensions to be compiled. This allows us to use the familiar Python syntax and its rich standard library in our code. Let's take a look at the Python Pexpect library.

Paramiko is a Python implementation of the SSHv2 protocol. Paramiko is the low-level SSH client behind the high-level automation framework Ansible for its network modules. We will cover Ansible in later chapters. Let's take a look at the Paramiko library.

We have taken a pretty huge leap forward in this chapter as far as automating our network using Python is concerned. However, the method we have used feels like somewhat of a workaround for automation. We attempted to trick the remote devices into thinking they were interacting with a human on the other end.

Let us summarize what we learned from this lesson.

This lesson discusses the newer network devices that support Application Programming Interfaces (APIs) and other high-level interaction methods. It also illustrates tools that allow abstraction of low-level tasks while focusing on the intent of the network engineers. A discussion about and examples of Cisco NX-API, Juniper PyEZ, and Arista Pyeapi will be used in the chapter.

In a perfect world, network engineers and architects who design and manage networks should focus on what they want the network to achieve instead of the device-level interactions. The infrastructure will be described in lines of code with the necessary software or framework enforcing that state. Let’s look at it.

Cisco Systems, the 800-pound gorilla in the networking space, have not missed out on the trend of network automation. Since this course focuses on Python and networking, we will scope this section to the main networking products. In particular, we will cover the following:

• Nexus product automation with NX-API

• Cisco NETCONF and YANG examples

• The Cisco application-centric infrastructure for the data center

• The Cisco application-centric infrastructure for the enterprise

the difference between a service provider's network needs compared to a cloud data center is that, traditionally, service providers aggregate more services into a single device. A good example would be Multiprotocol Label Switching (MPLS) that almost all major service providers provide but rarely adapt in the enterprise or data center networks. Juniper, as they have been very successful, has identified this need and excel at fulfilling the service provider requirements of automating. Let's take a look at some of Juniper's automation APIs.

Arista Networks have always been focused on large-scale data center networks. Like other vendors, you can interact with Arista devices directly via eAPI, or you can choose to leverage their Python library. We will see examples of both. We will also look at Arista's integration with the Ansible framework in later chapters.

Let us summarize what we learned from this lesson.

This lesson discusses the basics of Ansible, an open source, Python-based automation framework. Ansible moves one step further from APIs and focuses on declarative task intent. In this chapter, we will cover the advantages of using Ansible, its high-level architecture, and see some practical examples of Ansible with Cisco, Juniper, and Arista devices.

Ansible is a framework that can simplify the process of going from business logic to network commands. It can configure systems, deploy software, and orchestrate a combination of tasks. Ansible is written in Python and has emerged as one of the leading automation tools supported by network equipment vendors.

There are many infrastructure automation frameworks besides Ansible—namely Chef, Puppet, and SaltStack. Each framework offers its own unique features and models; there is no one right framework that fits all the organizations. In this section, we will list some of the advantages of Ansible over other frameworks and why it is a good tool for network automation.

The Ansible architecture consists of playbooks, plays, and tasks. Let’s look at it in more detail.

Ansible was originally made for managing nodes with full operating systems such as Linux and Windows before it was extended to support network equipment. You may have already noticed the subtle differences in playbooks that we have used so far for network devices, such as the lines of gather facts: false and connection: local; we will take a closer look at the differences in the following sections.

Let us summarize what we learned from this lesson.

This lesson builds on the knowledge in the previous chapter and covers the more advanced Ansible topics. We will cover conditionals, loops, templates, variables, Ansible Vault, and roles. It will also cover the basics of writing custom modules.

Ansible conditionals are similar to conditional statements in programming languages. In this section, we will discuss the when clause, which is supported for all modules, as well as unique conditional states that are supported in Ansible networking command modules.

Ansible provides a number of loops in the playbook, such as standard loops, looping over files, subelements, do-until, and many more. In this section, we will look at two of the most commonly used loop forms: standard loops and looping over hash values.

The Ansible template is an important tool that we will be using in our daily task, and we will spend more of this section exploring it. We will learn the syntax by gradually building up our playbook from simple to more complex.

You may have noticed that our playbooks are getting kind of long. We’re also repeating some of our configuration for things like credentials. Ansible has group vars and host vars to help us with this. Let us look at it in detail.

As you can see from the previous section, in most cases, the Ansible variable provides sensitive information such as a username and password. It would be a good idea to put some security measures around the variables so that we can safeguard against them. The Ansible Vault provides encryption for files so they appear in plaintext.

The best way to handle complex tasks is to break them down into smaller pieces. Of course, this approach is common in both Python and network engineering. In Python, we break complicated code into functions, classes, modules, and packages. In networking, we also break large networks into sections such as racks, rows, clusters, and datacentres. In Ansible, we can use roles and includes to segment and organize a large playbook into multiple files. Breaking up a large Ansible playbook simplifies the structure as each of the files focuses on fewer tasks. It also allows the sections of the playbook to be reused.

By now, you may get the feeling that network management in Ansible is largely dependent on finding the right module for your task. There is certainly a lot of truth in that logic. Modules provide a way to abstract the interaction between the managed host and the control machine; they allow us to focus on the logic of our operations. Up to this point, we have seen the major vendors providing a wide range of modules for Cisco, Juniper, and Arista. Let’s look at it.

Let us summarize what we learned from this lesson.

This lesson introduces several Python tools to help you secure your network. It will discuss using Scapy for security testing, using Ansible to quickly implement access lists, and using Python for network forensic analysis.

The devices being used in this chapter are a bit different from the previous chapters. In the previous chapters, we were isolating a particular device by focusing on the topic at hand. For this chapter, we will use a few more devices in our lab in order to illustrate the function of the tools that we will be using. The connectivity and operating system information are important as they have ramifications regarding the security tools that we will show later in this chapter. For example, if we want to apply an access list to protect the server, we need to know what the topology looks like and which direction the client is making their connections from. The Ubuntu host connections are a bit different than what we have seen so far, so please make reference back to this lab section when you see the example later if needed.

Scapy is a powerful Python-based interactive packet crafting program. Outside of some expensive commercial programs, very few tools can do what Scapy can do, to my knowledge. Let's take a look at the tool.

The network access lists are usually the first line of defense against outside intrusions and attacks. Generally speaking, routers and switches process packets at a much faster rate than servers, because they utilize hardware such as Ternary Content-Addressable Memory (TCAM). Let’s look at it in detail.

In this section, we will try to use Python to search through the Syslog text in order to detect the activities that we were looking for. Of course, the exact terms that we will search for depending on the device we are using. For example, Cisco provides a list of messages to look for in Syslog for any access list violation logging.

There are other network security tools that we can use and automate with Python. Let’s take a look at a few of them.

Let us summarize what we learned from this lesson.

This lesson covers monitoring the network using various tools. The lesson contains some examples using SNMP and PySNMP for queries to obtain device information. Matplotlib and Pygal examples will be shown for graphing the results. The lesson will end with a Cacti example using a Python script as an input source.

The lab for this chapter is similar to the one in Chapter 6, Network Security with Python, but with this difference: both of the network devices are IOSv devices. Let’s look at it.

SNMP is a standardized protocol used to collect and manage devices. SNMP is widely used in network monitoring and has been around since 1988 as part of RFC 1065. Let's jump directly into package installation and our first SNMP example.

In this section, we will use the data we collected from the last section using SNMP and use two popular Python libraries, Matplotlib and Pygal, to graph them.

Released in 2001, Cacti is an open source web-based network monitoring and graphing tool designed as an improved frontend for RRDtool. Cacti offers the custom data query feature that we can use Python for. In this section, we will see how we can use Python as an input method for Cacti.

Let us summarize what we learned from this lesson.

This lesson covers more network monitoring tools. The lesson will start with using Graphviz to graph the network from LLDP information. We will move to use examples with push-based network monitoring using Netflow and other technologies. We will use Python to decode flow packets and ntop to visualize the results. An overview of Elasticsearch and how it can be used for network monitoring will also be covered.

Graphviz is an open source graph visualization software. In this section, we will use the Link Layer Discovery Protocol (LLDP) to query the device neighbors and create a network topology graph via Graphviz. Upon completing this extensive example, we will see how we can take something new, such as Graphviz, and combine it with things we have already learned to solve interesting problems. Let's start by constructing the lab we will be using.

As mentioned in the chapter introduction, besides polling technology, such as SNMP, we can also use a push strategy, which allows the device to push network information toward the management station. Let’s look at it in more detail.

The last open source, general-purpose, distributed search and analytics engine called Elasticsearch is introduced in this section. It is often referred to as just Elastic or ELK stack for combining Elastic with the frontend and input packages Logstash, and Kibana, respectively.

Let us summarize what we learned from this lesson.