Introduction

Ever dreamed of a test framework that could be used across multiple platforms, OS and vendors, which could do regression, sanity and feature testing; already used by thousands of engineers and developers worldwide? Guess what, it exists, it’s free, and you can start using it right now!

pyATS (Python Automated Test Systems, to be pronounced “py A. T. S.”) was first created as an internal project, to ease the validation of two OS versions. It has been made public in 2017 through Cisco Devnet.

This blog post will be the third one of a series on pyATS. Today, we will explore pyATS libraries (also known as Genie), we will use a model to collect a sample output and understand when to use this capability.

More use cases are going to be covered in the next posts.

Other pyATS episodes

You’ve missed the first episode? You would like to read more? Below the list of published episodes:

pyATS librairies overview

pyATS building blocks

pyATS is made of three main building blocks:

• pyATS, the core block of this ecosystem. It’s a Python framework which leverages multiple Python libraries such as Unicon, providing a simplified connection experience to network devices. It supports CLI, NETCONF, RESTCONF and gRPC. It enables network engineers and developers to start with small and simple test cases.
• pyATS libraries (also known as Genie) which provides everything you need for network testing such as parsers, triggers and APIs.
• XPRESSO, the pyATS Web UI Dashboard.

You can read more about pyATS ecosystem in the official documentation.

pyATS libraries are the pyATS SDK (Software Development Kit, a big toolbox) which contains all the tools that you need for Network Test Automation. It has been used internally at Cisco since 2010 for our automated testing. Yes! You can use the same tools that we use to automate the validation of your network.

You can read more about pyATS librairies in the official documentation.

pyATS libraries in a nutshell

• 2800+ parsers accross 11 OS (as of April 2021),
• 32 supported models (more to come about them in a coming episode),
• Multiple tools for Test Harness such as triggers or traffic,
• Ansible and Robot libraries for easy integration with other tools.

You can find supported parsers and models in the official documentation.

Getting your hands dirty

Enough talking, let’s code!

pyATS installation has been covered in the First episode. Check it out to learn how to install pyATS.

Parsing a CLI output from a device

In the previous use case, we saw how we can collect and parse a CLI output (show ip interface brief). Today, we will see how to use pyATS models in order to retrieve a consistent output when polling devices running different OS.

In order for everyone to be able to run the code, we will use the IOS XR always-on sandbox on Cisco Devnet. Below the sandbox information.

Today, to demonstrate the Learn capability, we will use a sandbox running another operating system: IOS XE always-on sandbox on Cisco Devnet. Below, the sandbox information.

Feel free to adapt the code to use your own device(s).

Building a testbed

The simplest way to connect to a device is through a pyATS testbed file, written in YAML. This information will be used by Unicon to connect to the device and send the requested commands.

You can find the complete documentation on how to build a testbed here.

testbed.yaml

# Step 0: list of devices
devices:
  iosxr1:
    # Step 1: OS and Type
    type: iosxr-devnet
    os: iosxr
    # Step 2: credentials
    credentials:
      default:
        username: admin
        password: C1sco12345
    # Step 3: connection parameters
    connections:
      vty:
        protocol: ssh
        ip: sbx-iosxr-mgmt.cisco.com
        port: 8181
  csr1000v:
    # Step 1: OS and Type
    type: iosxe-devnet
    os: iosxe
    # Step 2: credentials
    credentials:
      default:
        username: developer
        password: C1sco12345
    # Step 3: connection parameters
    connections:
      vty:
        protocol: ssh
        ip: ios-xe-mgmt.cisco.com
        port: 8181

The testbed.yaml file is available here.

The testbed construction has been covered in the First episode. Have a look to understand how to build a testbed from scratch

Why do models exist?

Now, what if you have to collect this output from multiple devices running different OS? The output might be slightly different. On one operating system (OS), a key could be missing, renamed and/or require an additional command to be correctly populated.

Genie has a tool called Learn to accomplish validation accross multiple devices which could be running different OS.

For each feature, the operational information is collected by executing multiple show-commands, after which that output is parsed and stored into a Python datastructure. This structure will be the same for any OS supported by the model. Said differently, the Python structure between two OS supported by the model will have the same nested structure and the same keys (ex: description); but probably not the same values (ex: Configured using NETCONF!).

Below, a diagram of the show-commands sent to the device to fully populate the Interface model, for each supported OS.

You can find all pyATS supported models in the official documentation.

When to use models?

When should you be using Learn and when should you be using Parse? It all depends of your use case. The below lines should make you be able to choose the best tool, depending of the situation.

Use a Learn when:

• you have a testbed with multiple devices, running different OS,
• you want a single consistent output.

Use Parse when:

• you have a testbed with a single OS,
• you care about efficiency (remember, Learn will send multiple show-commands in order to always be consistent. It could take a long time if run of a big testbed.)

Collecting and parsing a CLI output with pyATS libraries models

Now that we understand when to use Learn (models) and when to use Parse, let’s look at the code. This script will collect the Inteface model of two devices. One running IOS XR, the other running IOS XE. It will extract the interface name and the interface ip address from the dictionary, then print each couple. The script will be further detailed and explained below.

2_multi_os.py

# New module! Now using Genie!
from genie import testbed

# Step 0: Load the testbed
testbed = testbed.load('./testbed.yaml')

# Step 1: Iterate through each device in the testbed
for device in testbed:

    print('')

    # Step 2: Connect on the device and print its name
    device.connect(init_exec_commands=[], init_config_commands=[], log_stdout=False)
    
    print('-----------------------------------')
    print('-- Connected on device: {device} --'.format(device=device.alias))
    print('-----------------------------------')

    # Step 3: Learn interface model from the device
    show_interface = device.learn('interface')

    for interface, info in show_interface.info.items():

        # Step 4: What if the key 'ipv4' doesn't exist (= no assigned IPv4)?
        if 'ipv4' in info:
            for ip, value in info['ipv4'].items():
                print('{interface} -- {ip}'.format(interface=interface, ip=value['ip']))
        else:
            print('{interface} -- Unassigned'.format(interface=interface))
    
    print('')

    # Step 5: Disconnect from the device
    device.disconnect()

The 2_multi_os.py file is available here.

Executing the script

From your bash terminal

python 2_multi_os.py

In this example, the testbed.yaml file need to be in the same folder as the 2_multi_os.py file. Also, you need to execute the Python script in the folder where you have these two files.

Let’s now explain the building blocks of the Python script. The parts below will refer to each inline comment of the code block above.

Output example

Here is an output example of the above script. It might slightly vary according to the configuration of the device.

Python console

---------------------------------
-- Connected on device: iosxr1 --
---------------------------------
Loopback200 -- 1.1.1.200
Loopback100 -- 1.1.1.100
GigabitEthernet0/0/0/6 -- Unassigned
GigabitEthernet0/0/0/5 -- Unassigned
GigabitEthernet0/0/0/4 -- Unassigned
GigabitEthernet0/0/0/3 -- Unassigned
GigabitEthernet0/0/0/2 -- Unassigned
GigabitEthernet0/0/0/1 -- Unassigned
GigabitEthernet0/0/0/0 -- Unassigned
MgmtEth0/RP0/CPU0/0 -- 10.10.20.175
Null0 -- Unassigned


-----------------------------------
-- Connected on device: csr1000v --
-----------------------------------
Loopback3 -- 8.8.8.8
GigabitEthernet3 -- 7.7.7.7
GigabitEthernet2 -- 10.10.10.1
GigabitEthernet1 -- 10.10.20.48
Loopback6 -- 99.99.48.20
Loopback5 -- 3.3.3.3

Step 0: load the testbed

From the genie module, we import the testbed.load() function. This function will be used to load the testbed file we have created.

We load the testbed information, stored in our testbed.yaml file. We assign it to an object: testbed.

Step 1: iterate through each device in the testbed

testbed object has an iterator capability. When used, it returns each device object in the testbed. We will iterate through each device in the testbed, connect and populate our model.

Step 2: connect to the device

We use the connect() method on the iosxr1 object to connect to the device.

By default, pyATS will send exec and configuration commands to the device (such as terminal length 0 and show version). To avoid such behavior, we are passing arguments to the conect() method. We are also disabling the logging to standard output. More information in the [documentation].(https://pubhub.devnetcloud.com/media/unicon/docs/user_guide/connection.html)

Step 3: learn interface model from the device

This step is the most important step in our script. It will Learn the interface model. pyATS will send multiple show-commands to the device in order to fully populate the model. Each information of the CLI output will be mapped either as a dictionary key or a value. Because we need to have the same keys between models, there could be entropy loss between the raw CLI output and the parsed output. For example, if an operational data is unique to an OS.

To do so, we are using the learn() method on the device object. The learn method takes a string as parameter, which is the model we would like to collect and parse. We are saving this parsed output in a variable show_interface.

You can find all pyATS supported models in the official documentation.

A parsed output example (i.e. the dictionary saved in the variable show_interface) can be seen in Step 4.

Step 4: Python logic to print interface name and IP

Below an example of parsed output for the interface model. Most interfaces are missing (only one is shown), for conciseness.

Parsed CLI output

{'GigabitEthernet0/0/0/0': {'bandwidth': 1000000,
                            'counters': {'in_broadcast_pkts': 0,
                                         'in_crc_errors': 0,
                                         'in_discards': 0,
                                         'in_multicast_pkts': 0,
                                         'in_octets': 0,
                                         'in_pkts': 0,
                                         'last_clear': 'never',
                                         'out_broadcast_pkts': 0,
                                         'out_errors': 0,
                                         'out_multicast_pkts': 0,
                                         'out_octets': 0,
                                         'out_pkts': 0,
                                         'rate': {'in_rate': 0,
                                                  'in_rate_pkts': 0,
                                                  'load_interval': 300,
                                                  'out_rate': 0,
                                                  'out_rate_pkts': 0}},
                            'enabled': False,
                            'encapsulation': {'encapsulation': 'arpa'},
                            'flow_control': {'flow_control_receive': False,
                                             'flow_control_send': False},
                            'ipv6': {'enabled': False},
                            'mac_address': '0050.56bb.4247',
                            'mtu': 1514,
                            'oper_status': 'down',
                            'phys_address': '0050.56bb.4247',
                            'type': 'gigabitethernet'}

In the above output, we have a list of interfaces: GigabitEthernet0/0/0/0 (others are not shown for conciseness). We are iterating through this list. For each interface, we are accessing the ip_address value. We’re then printting the interface name and IP.

The info attribute of the Interface object returns a Python dictionnary. We iterate through this dictionnary items. It returns all the tuples of key: values of this dictionnary.

It might not be useful in a real life use case. Goal here is to take a concise example, to show how easy it is to extract values of a CLI output when parsed with pyATS libraries.

Step 5: disconnect from the device

We use the disconnect() method to properly disconnect from the device.

It’s important to properly disconnect from the device, otherwise the vty connection will remain open on the device, until it times out.

Conclusion

In this second episode of the pyATS series, we learnt:

• The difference between pyATS Learn and Parse capabilities,
• Why pyATS models are powerful,
• How to collect and parse a CLI output from multiple devices using pyATS Learn,
• How to get a value from a parsed output.

In the next post, we will see our first use case: how to collect many show commands on many devices. Stay tunned!

The code used for each blog post can be found here. This link will include the code for all posts.

Resources

Below a few useful pyATS resources.

• List of supported pyATS parsers,
• The official pyATS documentation,
• List of Unicon supported platforms,
• Devnet code exchange,
• Join the Webex space with the pyATS community.