Introduction

Ever dreamed of a test framework that could be used across multiple platforms, OS and vendors, which could do regression, sanity and feature testing; already used by thousands of engineers and developers worldwide? Guess what, it exists, it’s free, and you can start using it right now!

pyATS (Python Automated Test Systems, to be pronounced “py A. T. S.”) was first created as an internal project, to ease the validation of two OS versions. It has been made public in 2017 through Cisco Devnet.

This is the 4th blog post of the pyATS series. Today, we will cover our first use case: how to collect many show commands on many devices?

Today’s code will be available here.

More use cases are going to be covered in the next posts.

Other pyATS episodes

You’ve missed the first episode? You would like to read more? Below the list of published episodes:

pyATS libraries in a nutshell

• 2800+ parsers accross 11 OS (as of April 2021),
• 32 supported models (more to come about them in a coming episode),
• Multiple tools for Test Harness such as triggers or traffic,
• Ansible and Robot libraries for easy integration with other tools.

You can find supported parsers and models in the official documentation.

Getting your hands dirty

Enough talking, let’s code!

pyATS installation has been covered in the First episode. Check it out to learn how to install pyATS.

Today’s code will be available here.

We will first explain the code in a high level view. We will then explain each building block individually.

In order for everyone to be able to run the code, we will use the IOS XR always-on sandbox on Cisco Devnet. Below the sandbox information.

Use case

I had this use case a couple of months ago. We had a thermal issue on a device. We wanted to know if the other devices (400) also suffered the same root cause. To verify, the TAC Engineer asked us to collect 4 show commands on 400 devices.

How long would it take to do it manually? 60 seconds per device?

Without optimizing the script (ex: using threads) it took us 20 minutes to collect such outputs. We were more than ready when the Engineer asked for a couple more commands.

High level code

The below diagram presents the high level code. In a nutshell:

• 0) extract the IP address of each device
• 1) generate the testbed using Jinja2
• 2) extract the show commands
• 3) python logic to collect each show command and write the output

For each device, we will have an output file with the show commands collected.

Extract the IP address of each device

The list of IP addresses is stored in templates/list_ip.yaml. We can add extra IP addresses by adding a new item in the yaml file, as below.

List example in YAML

- ip_1
- ip_2
- ip_n

To create a list out of a yaml file, we are using the PyYAML package.

Extract a list of IP from a YAML file

import yaml

with open("./templates/list_ip.yaml", "r") as file:
    list_ip = yaml.load(file, Loader=yaml.FullLoader)

You can find PyYAML documentation here.

Generate the testbed using Jinja2

Jinja2 is a templating engine. You can create a sample structure with keywords. Jinja2 will find and replace these keywords with your own values. For example, you could have this sample IOS XR interface template.

Jinja2 IOS XR interface template

In the above example, you would give Jinja2 three arguments: name, ipv6 and mask.

What’s interesting with Jinja2 is that it can have its own logic such as condition, loops and blocks. Using a Jinja2 loop, the above interface template could be reused multiple times. You could loop 50 times, to create 50 loopbacks with a unique ID (name) and a unique IPv6 address.

Here, we are using Jinja2 to create a template for our pyATS testbed. You can find the pyATS testbed template in templates/testbed.tpl.

The testbed construction has been covered in the First episode. Have a look to understand how to build a testbed from scratch.

The below outputs presents the Jinja2 logic used for our pyATS testbed. For brievity, we are not showing the full template file.

Jinja2 logic with pyATS testbed

We are giving Jinja2 a list of lists: list_ip_id. Each sub-list contains the ip address of a device and a unique id to identify the node’s name in the testbed. This value has to be unique. For each list in list_ip_id we will create a new node id and populate its ip.

Now, we need to write Python logic to give this list_ip_id to Jinja2 template. That’s how you do it.

Python logic to populate Jinja2 template

# Where's the folder with my templates (or my folders, if multiple)
template_loader = jinja2.FileSystemLoader(searchpath="./templates")

# Instance of the Environment class. Gives the loader (above), optionally parameters like
# block strings, variable strings etc.
template_env = jinja2.Environment(loader=template_loader)

# Which file is my template
template = template_env.get_template("testbed.tpl")

# We give the template two lists:
# - list_ip: the IP of our devices
# - range(len(list_ip)), the id (from 0 to the max device) that will be used in device.name to make it unique
testbed = load(template.render(list_ip_id = zip(list_ip, range(len(list_ip)))))

In Python, the zip() function takes iterables (can be zero or more), aggregates them in a tuple, and returns it. More information here.

The testbed information is specific to the Devnet sandbox: login, password, protocol, operating system… Feel free to change it.

Extract the list of show commands

The list of IP addresses is stored in templates/list_show.yaml. Same as before, we are using PyYAML to create a list out of a yaml file.

Collect and write the outputs

Connect to each device

First, we need to connect to each device. In case we cannot connect to a device, Unicon will send a ConnectionError. We are catching such error to print in the terminal if we cannot connect to a device. In the below logic, if we cannot connect to a device, it doesn’t fail the script. We will still iterate through the other devices, as long as we have devices in the testbed. Feel free to change this behavior if needed.

Python logic to connect to each device

for device in testbed:
    
    try:
        device.connect(learn_hostname=True,
                    init_exec_commands=[],
                    init_config_commands=[],
                    log_stdout=False)
    except ConnectionError:
        print("-- ERROR --")
        print(f"  Can't connect to {device.connections.vty.ip}")
        continue

The connect() method has been covered in the First episode. Have a look to understand how it works.

Collecting CLI output

Last, we need to collect each CLI output and write it in a file. File name will be the device’s hostname (learned with pyATS).

In case a command is invalid, Unicon will send a SubCommandFailure. We are cathing this error, to tell in the terminal which show command failed. We will still iterate through the other show commands, as long as we have show commands in the list_show. Feel free to change this behavior if needed.

with open(f'./outputs/{device.hostname}.txt', 'w') as file:

        # Collect and write each output
        for show in list_show:
            file.write(f'--- {show} ---\n')
            try:
                file.write(device.execute(show))
                file.write('\n\n')
            except SubCommandFailure:
                print(f'  /!\ `{show}` invalid command. Skipping.')

Conclusion

In this fourth episode of the pyATS series, we learnt:

• How to extract a list from a YAML file,
• How to use Jinja2 logic to generate a testbed,
• How to catch exceptions to modify the default python behavior.

In the next post, we will learn more about pyATS Dq (dictionnary querry) and how pyATS can find a pair of key: value in a nested structure. Yes, it even supports regex!

Resources

Below a few useful pyATS resources.

• List of supported pyATS parsers,
• The official pyATS documentation,
• List of Unicon supported platforms,
• Devnet code exchange,
• Join the Webex space with the pyATS community.