{ "blogs-2016-10-14-ios-xr-packages-and-security": { "title": "IOS-XR Packages and Security", "content": " IOS-XR Packages and Security Introduction Packages Verification Package Signature Example for the puppet client IntroductionWith the Introduction of IOX XR 6.0, the complete IOS XR software architecture has migrated to a open source infrastructure centered around the Linux Operating System. With the Adoption of Linux and Linux Containers (LXCs) some major change have been introduce in several areas; The monolithic set of XR features has been dis-aggregated in a collection of RPM packages that can be upgraded trough local or remote repositories. The Linux environment comes with its own authentication mechanism and user privileges. With XR and Linux processes residing side-by-side inside the same namespace, it is important to control the installation and execution of these processes.Packages VerificationPackages and ISOs are delivered in a tar file. included in that tar file is a readme file that contain a MD5 sum for all the files. MD5 sum are a good way to verify the integrity of the file using the md5sum utility on Unix. md5sum generate a 128-bit value known as the “digital fingerprint.” If two files have different MD5 sums, the files are definitely different whereas if two files have the same MD5 sum, it is highly likely the files are exactly alike. MD5 is technically a cryptography hash function which essentially means that the risk of producing collisions is really low but possible.Example of verifying md5sum from the Linux shell using unnamed pipes.cisco@galaxy-42#~$ diff -awsy <(grep mgbl README-ncs5k-k9sec-6.0.0 | cut -d' ' -f1) <(md5sum ncs5k-mgbl-2.0.0.0-r600.x86_64.rpm-6.0.0 | cut -d' ' -f1)7a0e22ea86622dfc2293179d6fb52721 7a0e22ea86622dfc2293179d6fb52721Files /dev/fd/63 and /dev/fd/62 are identicalor using a simple script#!/bin/bashif [ -z $3 ]; then\techo ~file + readme + feature needed~\techo ~usage# 0? ~\texitfiexport csum=$(grep $3 $2 | cut -d' ' -f1)export fsum=$(md5sum $1 | cut -d' ' -f1)echo $csumecho $fsumif [ ~$csum~ = ~$fsum~ ]; then\techo ~MD5sum are identical~else\techo ~MD5sum are different~ficisco@galaxy-42#~$ chkmd5.sh ncs5k-mgbl-2.0.0.0-r600.x86_64.rpm-6.0.0 README-ncs5k-k9sec-6.0.0 mgbl7a0e22ea86622dfc2293179d6fb527217a0e22ea86622dfc2293179d6fb52721MD5sum are identicalPackage SignatureIOS XR 6.0.0 and above use the RPM format for all its packages, the RPM format include the capability to digitally sign packages using a SHA-1 key. At this time none of the IOS XR packages are signed, this feature will be available in a future release.Some third party packages embed a RSA/SHA-1signature, to install these packages, the public key of the provider can be verified before and during package installation.Example for the puppet client1) Key PresenceOn the local repository server, you can verify if the package has been signed by the provider.cisco@galaxy-42#~$ rpm -qpi puppet-agent-1.4.1-1.cisco_wrlinux7.x86_64.rpm | grep Signaturewarning# puppet-agent-1.4.1-1.cisco_wrlinux7.x86_64.rpm# Header V4 RSA/SHA1 Signature, key ID 4bd6ec30# NOKEYSignature # RSA/SHA1, Thu 24 Mar 2016 03#05#33 PM PDT, Key ID 1054b7a24bd6ec302) Key VerificationOn the local repository download and import the public key from the package provider, this step can be performed multiple time to verify the integrity of all packages in the local repository.cisco@galaxy-42#~$ wget http#//yum.puppetlabs.com/RPM-GPG-KEY-puppetlabscisco@galaxy-42#~$ rpm --import RPM-GPG-KEY-puppetlabcisco@galaxy-42#~$ rpm -Kv puppet-agent-1.4.1-1.cisco_wrlinux7.x86_64.rpmpuppet-agent-1.4.1-1.cisco_wrlinux7.x86_64.rpm# Header V4 RSA/SHA1 Signature, key ID 4bd6ec30# OK Header SHA1 digest# OK (fd954ea78e24ea32fdbaf3be045087ffe4c277ae) V4 RSA/SHA1 Signature, key ID 4bd6ec30# OK MD5 digest# OK (5eb0292058ba82449b7fb6eaa62fc102)3) Create a Repository PointerOn the Router, create a .repo file in /etc/yum/repos.d that enable key verification of packages, keys can also be copied on a local repository if this repository is secure.[puppetlabs]name=puppetlabsbaseurl=http#//galaxy-42.cisco.com/Packagesgpgcheck=1gpgkey=http#//yum.puppetlabs.com/RPM-GPG-KEY-puppetlabsenabled=14) Package Installation from local repositoryxr-vm_node0_RP0_CPU0#~# yum install puppet-agentLoaded plugins# app_plugin, downloadonly, protect-packages puppetlabs | 2.9 kB 00#00 Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package puppet-agent.x86_64 0#1.4.1-1.cisco_wrlinux7 will be installed --> Finished Dependency Resolution Dependencies Resolved ====================================================================================================== Package Arch Version Repository Size ====================================================================================================== Installing# puppet-agent x86_64 1.4.1-1.cisco_wrlinux7 puppetlabs 41 M Transaction Summary ====================================================================================================== Install 1 Package Total download size# 41 M Installed size# 143 M Is this ok [y/N]# y Retrieving key from http#//yum.puppetlabs.com//RPM-GPG-KEY-puppetlabs Importing GPG key 0x4BD6EC30# Userid# ~Puppet Labs Release Key (Puppet Labs Release Key) <info@puppetlabs.com>~ From # http#//yum.puppetlabs.com//RPM-GPG-KEY-puppetlabs Is this ok [y/N]# y Downloading Packages# puppet-agent-1.4.1-1.cisco_wrlinux7.x86_64.rpm | 41 MB 00#01 Running Transaction Check Running Transaction Test Transaction Test Succeeded Running Transaction Installing # puppet-agent-1.4.1-1.cisco_wrlinux7.x86_64 1/1 Installed# puppet-agent.x86_64 0#1.4.1-1.cisco_wrlinux7 Complete!", "url": "/blogs/2016-10-14-ios-xr-packages-and-security/", "author": "Patrick Warichet", "tags": "iosxr, cisco, linux, RPM" } , "blogs-2016-10-17-ios-xr-users-and-groups-inside-linux": { "title": "IOS-XR users and groups inside Linux", "content": " IOS-XR# Linux users and groups XR and Linux Users Gaining Root Privilege XR and Linux UsersBy default, any user created inside XR is automatically replicated including that user’s password inside Linux using a unique UID and GID.This allows basic access into the Linux shell for all XR configured user, the administrator can create multiple users directly from the XR console if desired.Inside Linux, nine special groups are created by default, each of these groups maps to one of the default XR groups. When the administrator creates a user belonging to one of the default XR group, that user get replicated inside Linux and added to that special Linux group. In addition, XR users that are member of certain default group are granted root access to Linux when they issue the “bash” or “run” command (see table below). XR Group Linux Group GID Role Access Linux from XR cisco-support cisco-support 1000 Cisco support personnel tasks n/a (add-on group for root-lr users) maintenance maintenance 1001   Yes netadmin netadmin 1002 Network administrator tasks No provisioning provisioning 1003   Yes retrieve retrieval 1004   No root-lr root-lr 1005 Secure domain router administrator tasks Yes n/a root-system 1006 System-wide administrator tasks Compatibility with previous IOS-XR serviceadmin serviceadmin 1007 Service administration tasks No sysadmin sysadmin 1008 System administrator tasks No operator operator 37 Operator day-to-day tasks (demo) No In the following example, the administrator creates a user “test10” member of the maintenance group, which allow the user to enter the Linux shell as root (uid/gid# 0) via the run command, the user “test10” (uid# 1010) is member of the group “test10” (gid#1019) and secondary group maintenance (gid#1001)RP/0/RP0/CPU0#pod-rtr(config)#username test10 group maintenanceRP/0/RP0/CPU0#pod-rtr(config)#username test10 password test10RP/0/RP0/CPU0#pod-rtr(config)#commitRP/0/RP0/CPU0#pod-rtr(config)#endRP/0/RP0/CPU0#pod-rtr#exit...Username# test10Password#RP/0/RP0/CPU0#pod-rtr#bashTue Mar 29 03#06#41.759 UTC[xr-vm_node0_RP0_CPU0#~]$iduid=0(root) gid=0(root) groups=0(root)[xr-vm_node0_RP0_CPU0#~]$id test10uid=1010(test10) gid=1019(test10) groups=1019(test10),1001(maintenance)Gaining Root PrivilegeInside Linux the file /etc/sudoers only allows root to do everything on any machine as any user ~root ALL=(ALL) ALL~. No other user can gain root privilege by default, The administrator will have to modify the /etc/sudoers as root to allow other users to gain root access via the sudo command.These measures ensure that only the users with access to the “run” or “bash” command can create initial users in the Linux shell and provide sudo access.A common practice is to allow all members of the sudo group (GID# 27) root privileges. This is done by un-commenting the line ~#%sudo ALL=(ALL) ALL~ as root in the “/etc/sudoers” file and add users to the sudo group.[xr-vm_node0_RP0_CPU0#~]$usermod -a -G sudo test10[xr-vm_node0_RP0_CPU0#~]$id test10uid=1010(test10) gid=1019(test10) groups=1019(test10),27(sudo),1001(maintenance)After the line has been modified any user member of the sudo group can gain root privilege using the sudo command. Sudo logs each command, providing a clear audit trail of who did what, Sudo uses timestamp files to implement a “ticketing” system. When a user invokes sudo and enters their password, they are granted a ticket for 5 minutes. for more information on sudoers visit sudo main pageUpdate# Since 6.1.1 All members of the the sudo group can gain root privilege using “sudo” by default.", "url": "/blogs/2016-10-17-ios-xr-users-and-groups-inside-linux/", "author": "Patrick Warichet", "tags": "iosxr, cisco, linux" } , "blogs-2016-10-20-using-ztp-to-install-puppet": { "title": "Using ZTP to install Puppet", "content": " Using ZTP to install Puppet Introduction Puppet in IOS-XR Script Example IntroductionPuppet is a configuration management tool for information technology (IT) professionals. Puppet can potentically manage any resource defined on a node. Puppet can manage complex and distributed components to ensure service consistency and availability. In short, Puppet uses a configuration policy referred to as a “recipe” to bring systems into compliance.Puppet enables you to make a lot of changes both quickly and consistently. Unlike scripts, you don’t have to write out every step, you only have to define how it should be. You are not required to write out the process for evaluating and verifying different conditions. Instead, you utilize the Puppet configuration language to declare the final state of the resources. This is why Puppet is often described as a declarative language.The difference between a declarative language and a procedural language is# You tell Puppet the desired end results, but not the steps to get there.If you choose to use Puppet as your configuration platform, it is important to have the agent installed on the devices early on in the deployment process, and not use any scripting beyond the installation procedure of the agent.Puppet in IOS-XRPuppetlabs have created a puppet agent for the IOS-XR Yocto based Linux. You can follow their installation instructions here#Installing Cisco IOS-XR agents. Make sure you download the file which has “cisco-wrlinux-7” in the filename, since this is the version that works in the IOS-XR Yocto Linux environment.If you are interested in using Puppet in conjunction with YANG data models, take a look at the tutorial on Puppet and ciscoyang Using Puppet with IOS-XR 6.1.1. You will have to modify the example below to include the installation of the GRPC GEM file.Script ExampleHere is a small example on how to use ZTP to download and install the Puppet package from a local repository, configure the device hostname and enable DNS resolution. The Puppet agent package is secured with a SHA-1 (GPG) key, and you will need to download it to a secure location or access the key directly from Puppetlabs. In the example below, I simply placed the key in the same directory as the RPM package.Puppet requires all hostnames to be resolved. The script relies on a DNS server to resolve the hostname and include the modifications to IOS-XR Linux. The script also assumes that communication between the Puppet master and the agent will happen over the management interface.#!/bin/bashYUM_REPO=~http#//172.30.0.22/packages/puppet~YUM_PUPPET=~/etc/yum/repos.d/puppet.repo~PUPPET_SRV=~puppet-master.cisco.local~PUPPET_CONF=~/etc/puppetlabs/puppet/puppet.conf~DOMAIN=~cisco.local~DOMAIN_SRV=172.30.0.25HOSTNAME=~ncs-5001-c~MGMT_IP=~172.30.12.54 255.255.255.0~source ztp_helper.shfunction create_repo(){ # Create local repository file for puppet echo ~### created by ztp $(date +~%b %d %H#%M#%S~) ###~ > $YUM_PUPPET echo -ne ~[puppetlabs]\\nname=puppetlabs\\nenabled=1\\ngpgcheck=1\\n~ >> $YUM_PUPPET echo ~baseurl=$YUM_REPO~ >> $YUM_PUPPET echo ~gpgkey=$YUM_REPO/RPM-GPG-KEY-puppetlabs~ >> $YUM_PUPPET }function install_puppet(){ # Install puppet /usr/bin/yum clean all > /dev/null /usr/bin/yum update > /dev/null /usr/bin/yum install -y puppet > /dev/null}function config_puppet(){ echo ~### created by ztp $(date +~%b %d %H#%M#%S~) ###~ > $PUPPET_CONF echo -ne ~[main]\\nserver = $PUPPET_SRV\\n~ >> $PUPPET_CONF}function setup_resolver(){ local resolver=/etc/resolv.conf echo ~### created by ztp $(date +~%b %d %H#%M#%S~) ###~ > $resolver echo ~domain $DOMAIN~ >> $resolver echo ~search $DOMAIN~ >> $resolver echo ~nameserver $DOMAIN_SRV~ >> $resolver }function set_hostname(){ xrapply_string_with_reason ~ztp puppet install~ ~hostname $HOSTNAME\\n interface mgmtEth 0/RP0/CPU0/0\\n ipv4 address $MGMT_IP \\n~ /bin/hostname -f $HOSTNAME.$DOMAIN echo $HOSTNAME > /etc/hostname }function start_services(){ /etc/init.d/sshd_operns start /etc/init.d/puppet start}### script startset_hostname;setup_resolver;create_repo;install_puppet;config_puppet;start_services;exit 0", "url": "/blogs/2016-10-20-using-ztp-to-install-puppet/", "author": "Patrick Warichet", "tags": "iosxr, linux, puppet, ZTP" } , "blogs-2016-10-24-using-ztp-to-install-chef": { "title": "Using ZTP to install Chef", "content": " Using ZTP to install chef Introduction Chef Infrastructure Installing the Chef Server Create a User and Organization Installing and Setting up the Chef Workstation Installing the client with ZTP and Bootsrap the node Creating a simple recipe IntroductionChef is an automation platform that “turns infrastructure into code”, allowing users to manage and deploy resources across multiple servers, or nodes. Chef allows users to create and download recipes (stored in cookbooks) to automate content, configuration and policies on these nodes.Chef is comprised of a Chef server, one or more workstations, and a number of nodes that are managed by the chef-client installed on each node. You can download the IOS-XR Chef client package from Chef and installed directly inside the control plane LXC of IOS-XR.Chef like Puppet (see my previous blog Using ZTP to install Puppet)Automated configuration management tools play a vital role in managing complex enterprise infrastructures. Amongst the many advantages, the ones pertinent to IOS-XR and network node in general are#Consistency# It makes it easier for configuration changes to meet compliance and security requirements. By automating repeated tasks (like applying a SMU), it allows network administrators to concentrate on more important stuff.Efficient change management#. Automated configuration management can remove delay when deploying new technologies, reducing the number of processes needed to manage change. Small change batches can be performed on a more regular basis.Availability# Automatated configuration management tool help quickly restore service. Rather than troubleshooting an issue by hand, a system can be reset to well known working status.Visibility# Configuration management tools include auditing and reporting capabilities, changes can be automatically logged in all relevant tracking systems.Chef InfrastructureChef requires the follwing components a server, one or more workastation and one or more nodes, The instruction below used Ubuntu Xenial for both the server and the workstation to mange the IOS-XR nodes.Installing the Chef ServerThe Chef server is the central place that govern interaction between all workstations and managed nodes. Changes made in the workstations are uploaded to the Chef server, which is then accessed by the chef-client and used to configure individual nodes.Installing Chef Server is easy as 1-2-3#1 Download the latest Chef Server for your favorite distro#Example for Ubuntu Xenial (Chef version 12.9.1)wget https#//packages.chef.io/stable/ubuntu/16.04/chef-server-core_12.9.1-1_amd64.deb2 Install the server#sudo dpkg -i chef-server-core_*.deb3 Run the chef-server-ctl command to start the Chef server services#sudo chef-server-ctl reconfigureCreate a User and Organization1 In order to link workstations and nodes to the Chef server, an administrator and an organization need to be created with associated RSA private keys. create a directory to store the keys#mkdir ~/chef-keys2 Create an administrator. Change username to your desired username, firstname and lastname to your first and last name, email to your email, password to a secure password, and username.pem to your username followed by .pem#sudo chef-server-ctl user-create username firstname lastname email password --filename ~/chef-keys/username.pem3 Create an organization. The shortname value should be a basic identifier for your organization with no spaces, whereas the fullname can be the full, proper name of the organization. The association_user value username refers to the username made in the step above#sudo chef-server-ctl org-create shortname fullname --association_user username --filename ~/chef-keys/shortname.pemWith the Chef server installed and the RSA keys generated, you can move on to configuring your workstation, where all major work will be performed for your Chef’s nodes.Installing and Setting up the Chef WorkstationYour Chef workstation will be where you create and configure any recipes, cookbooks, attributes, and other changes made to your Chef configurations. Although this can be the same machine that host the server, it is recommended to keep the server and the workstation seperated.1 Download the latest Chef Development Kit#wget https#//packages.chef.io/stable/ubuntu/12.04/chefdk_0.19.6-1_amd64.deb2 Install ChefDK#sudo dpkg -i chefdk_*.deb3 Verify the components of the development kit#~$ chef verifyRunning verification for component 'berkshelf'Running verification for component 'test-kitchen'Running verification for component 'tk-policyfile-provisioner'Running verification for component 'chef-client'Running verification for component 'chef-dk'Running verification for component 'chef-provisioning'Running verification for component 'chefspec'Running verification for component 'generated-cookbooks-pass-chefspec'Running verification for component 'rubocop'Running verification for component 'fauxhai'Running verification for component 'knife-spork'Running verification for component 'kitchen-vagrant'Running verification for component 'package installation'Running verification for component 'openssl'Running verification for component 'inspec'Running verification for component 'delivery-cli'Running verification for component 'git'Running verification for component 'opscode-pushy-client'Running verification for component 'chef-sugar'.................---------------------------------------------Verification of component 'test-kitchen' succeeded.Verification of component 'chef-dk' succeeded.Verification of component 'chefspec' succeeded.Verification of component 'rubocop' succeeded.Verification of component 'knife-spork' succeeded.Verification of component 'openssl' succeeded.Verification of component 'delivery-cli' succeeded.Verification of component 'opscode-pushy-client' succeeded.Verification of component 'berkshelf' succeeded.Verification of component 'fauxhai' succeeded.Verification of component 'inspec' succeeded.Verification of component 'chef-sugar' succeeded.Verification of component 'tk-policyfile-provisioner' succeeded.Verification of component 'chef-provisioning' succeeded.Verification of component 'kitchen-vagrant' succeeded.Verification of component 'git' succeeded.Verification of component 'chef-client' succeeded.Verification of component 'package installation' succeeded.Verification of component 'generated-cookbooks-pass-chefspec' succeeded.4 Generate the chef-repo and add the RSA keys~$ chef generate repo chef-repo~$ cd chef-repo~/chef-repo$ mkdir .chef~/chef-repo$ scp user@chef-server#~/chef-keys/*.pem .chef/5 Generate knife.rbUsing your favorite text editor create a knife configuration file named knife.rb in to your ~/chef-repo/.chef folder.log_level #infolog_location STDOUTnode_name 'username'client_key '/home/cisco/chef-repo/.chef/username.pem'validation_client_name 'shortname-validator'validation_key '/home/cisco/chef-repo/.chef/shortname.pem'chef_server_url 'https#//chef-server/organizations/shortname'syntax_check_cache_path '/home/cisco/chef-repo/.chef/syntax_check_cache'cookbook_path [ '/home/cisco/chef-repo/cookbooks' ]Replace username,shortname with the values used in the steps “Create a User and Organization”Move uo to the chef-repo and copy the needed SSL certificates from the server#~/chef-repo/.chef$ cd ..~/chef-repo$ knife ssl fetchWARNING# Certificates from chef-cook will be fetched and placed in your trusted_certdirectory (~/chef-repo/.chef/trusted_certs).Knife has no means to verify these are the correct certificates. You shouldverify the authenticity of these certificates after downloading.Adding certificate for chef-cook in ~/chef-repo/.chef/trusted_certs/chef-cook.crtConfirm that knife.rb is set up correctly by running the client list#~/chef-repo$ knife client listciscolab-validatorThis command should output the validator name (ciscolab in our case).With both the server and a workstation configured, it is possible to bootstrap your first node.Installing the client with ZTP and Bootsrap the nodeUsing ZTP we can install the Chef client directly inside the control plane LXC of IOS-XR, here is a script example that will perform the installation during the initial bootup.#!/bin/bashYUM_REPO=~http#//172.30.0.22/packages/chef~YUM_CHEF=~/etc/yum/repos.d/chef.repo~CHEF_SRV=~chef-cook.cisco.local~DOMAIN=~cisco.local~DOMAIN_SRV=172.30.0.25HOSTNAME=~ncs-5001-c~MGMT_IP=~172.30.12.54 255.255.255.0~source ztp_helper.shfunction create_repo(){ # Create local repository file for chef echo ~creating repo file in /etc/yum/repo.d~ echo ~### created by ztp $(date +~%b %d %H#%M#%S~) ###~ > $YUM_CHEF echo -ne ~[chef]\\nname=chef\\nenabled=1\\ngpgcheck=1\\n~ >> $YUM_CHEF echo ~baseurl=$YUM_REPO~ >> $YUM_CHEF echo ~gpgkey=$YUM_REPO/chef.asc~ >> $YUM_CHEF }function install_chef(){ # Install chef from local repository echo ~installing chef from the local repo~ /usr/bin/yum clean all > /dev/null /usr/bin/yum update > /dev/null /usr/bin/yum install -y chef > /dev/null}function setup_resolver(){ echo ~ setting up the resolver~ local resolver=/etc/resolv.conf echo ~### created by ztp $(date +~%b %d %H#%M#%S~) ###~ > $resolver echo ~domain $DOMAIN~ >> $resolver echo ~search $DOMAIN~ >> $resolver echo ~nameserver $DOMAIN_SRV~ >> $resolver }function set_hostname(){ echo ~setting up the device hostname~ xrapply_string_with_reason ~ztp chef install~ ~hostname $HOSTNAME\\n interface mgmtEth 0/RP0/CPU0/0\\n ipv4 address $MGMT_IP \\n~ /bin/hostname -f $HOSTNAME.$DOMAIN echo $HOSTNAME > /etc/hostname }function start_services(){ echo ~starying services~ /etc/init.d/sshd_operns start # Start chef client in daemon mode and schedule a run every 5 min /usr/bin/chef-client -daemonize -i 300 -L /var/log/chef.log}### script startset_hostname;setup_resolver;create_repo;install_chef;start_services;exit 0On the workstation, we bootstrap the node using knife. It is important to note that the Chef client will run inside the Linux shell. Inside the Linux shell the default port for the ssh server is 57722 and ssh using the root user is disabled. Fortunatly IOS-XR root-lr users are not root when they ssh into the system (see my previous blog IOS-XR Users and Groups). Since we need root access to bootstrap the client we have to use –sudo and provide the user password.~$ knife bootstrap 172.30.12.54 --sudo -p 57722 -x admin -P cisco123 --node-name ncs-5001-cConnecting to 172.30.12.54172.30.12.54 knife sudo password# Enter your password# 172.30.12.54 172.30.12.54 -----> Existing Chef installation detected172.30.12.54 Starting the first Chef Client run...172.30.12.54 Starting Chef Client, version 12.15.19172.30.12.54 [2016-11-02T23#09#51+00#00] WARN# [inet] no ip address on fwdintf172.30.12.54 Creating a new client identity for ncs-5001-c using the validator key.172.30.12.54 resolving cookbooks for run list# []172.30.12.54 Synchronizing Cookbooks#172.30.12.54 Installing Cookbook Gems#172.30.12.54 Compiling Cookbooks...172.30.12.54 [2016-11-02T23#09#53+00#00] WARN# Node ncs-5001-c has an empty run list.172.30.12.54 Converging 0 resources172.30.12.54 172.30.12.54 Running handlers#172.30.12.54 Running handlers complete172.30.12.54 Chef Client finished, 0/0 resources updated in 04 secondsknife node listncs-5001-cCreating a simple recipeWe use the command “chef generate cookbook” to configure our cookbook, once created, we change to the recipes folder and edit the default.rb recipe.~/chef-repo/cookbooks$ chef generate cookbook ios-xrGenerating cookbook ios-xr- Ensuring correct cookbook file content- Ensuring delivery configuration- Ensuring correct delivery build cookbook contentYour cookbook is ready. Type `cd ios-xr` to enter it.There are several commands you can run to get started locally developing and testing your cookbook.Type `delivery local --help` to see a full list.Why not start by writing a test? Tests for the default recipe are stored at#test/recipes/default_test.rbIf you'd prefer to dive right in, the default recipe can be found at#recipes/default.rbWe create a simple recipe that will create a file in the home directory of the user (/disk0# for IOS-XR)~/chef-repo/cookbooks$ cd ios-xr/recipes/~/chef-repo/cookbooks$ vi default.rb## Cookbook Name## ios-xr# Recipe## default## Copyright 2016, YOUR_COMPANY_NAME## All rights reserved - Do Not Redistribute#file ~#{ENV['HOME']}/chef.txt~ do content 'Hello from Chef'endWe push the recipe to the Chef server and add it to the run list for the ncs-5001-c node~/chef-repo/cookbooks/ios-xr/recipes$ knife cookbook upload ios-xrUploading ios-xr [0.1.0]Uploaded 1 cookbook.~/chef-repo/cookbooks/ios-xr$ knife node run_list set ncs-5001-c 'recipe[ios-xr##default]'ncs-5001-c# run_list# recipe[ios-xr##default]The execution will occur within the interval configured (300 sec in our case), Here is what the logs look like#Starting Chef Client, version 12.15.19resolving cookbooks for run list# [~ios-xr##default~]Synchronizing Cookbooks# - ios-xr (0.1.0)Installing Cookbook Gems#Compiling Cookbooks...Converging 1 resourcesRecipe# ios-xr##default * file[/disk0#/chef.txt] action create - create new file /disk0#/chef.txt - update content in file /disk0#/chef.txt from none to ba4fda --- /disk0#/chef.txt\t2016-11-09 22#04#29.356188978 +0000 +++ /disk0#/.chef-chef20161109-16571-14v6aep.txt\t2016-11-09 22#04#29.355188978 +0000 @@ -1 +1,2 @@ +Hello from ChefRunning handlers#Running handlers completeChef Client finished, 1/1 resources updated in 03 seconds", "url": "/blogs/2016-10-24-using-ztp-to-install-chef/", "author": "Patrick Warichet", "tags": "iosxr, cisco" } , "blogs-2017-09-21-ios-xr-ztp-learning-through-packet-captures": { "title": "IOS-XR ZTP: Learning through Packet Captures", "content": " On This Page Introduction DHCPv4 using BOOTP filename Packet Captures DHCP Discover (Router to DHCP Server) DHCP Offer (DHCP Server to Router) DHCPv4 using Option 67 bootfile-name (IOS-XR Release 6.2.25+) Packet Captures DHCP Discover (Router to DHCP Server) DHCP Offer (DHCP Server to Router) DHCPv6 using Option 59 Packet Captures DHCPv6 SOLICIT (Router to DHCPv6 Server) DHCPv6 Advertise (DHCP Server to Router) IntroductionZero Touch Provisioning is quite often considered the cornerstone of web scale deployment practices. It brings forth a mindset that is quite emblematic of the way Large Scale (Web) Service providers provision their environment#       “The network device should never have to be configured manually through its console - from the instant that it is powered on right up until the services are brought up and telemetry data starts flowing.”There are dozens of techniques that vendors have brought forward over the years to enable users to push a configuration down to their devices often using on-premise DHCP servers and encoding the communication with the device within DHCP options. Over the years as devices became natively scriptable, a script of the user’s choice may be downloaded to automate provisioning of the entire system (NOS, agents, scripts etc.) on the whole and not just the configuration of the Network OS.IOS-XR is no exception and as Patrick’s excellent tutorial illustrates# https#//xrdocs.github.io/software-management/tutorials/2016-08-26-working-with-ztp/a lot of headway has been made in enabling IOS-XR to be completely scriptable as soon as it is powered on.To understand the variety of options available to classify the router when it first identifies itself to the DHCP server and to supply the required script, let’s look at packet captures in the following scenarios# DHCPv4 using BOOTP filename to supply script/config location DHCPv4 using Option 67 (bootfile-name) to supply script/config location. DHCPv6 using Option 59 (OPT_BOOTFILE_URL) to supply script/config location. The packet captures of complete DHCP transactions along with a sample isc-dhcp server config for each of the above cases has been made available on Github here# https#//github.com/ios-xr/ztp-pcapDHCPv4 using BOOTP filenameFor this purpose we use the the DHCPv4 server configuration located here.The config is reproduced here for the reader’s convenience#option space cisco-vendor-id-vendor-class code width 1 length width 1;option vendor-class.cisco-vendor-id-vendor-class code 9 = {string};######### Network 11.11.11.0/24 ################shared-network 11-11-11-0 {####### Pools ############## subnet 11.11.11.0 netmask 255.255.255.0 { option subnet-mask 255.255.255.0;\toption broadcast-address 11.11.11.255;\toption routers 11.11.11.2;\toption domain-name-servers 11.11.11.2;\toption domain-name ~cisco.local~;\t# DDNS statements \tddns-domainname ~cisco.local.~;\t# use this domain name to update A RR (forward map) \tddns-rev-domainname ~in-addr.arpa.~; \t# use this domain name to update PTR RR (reverse map) }######## Matching Classes ########## class ~ncs5508~ { match if (substring(option dhcp-client-identifier,0,11) = ~FGE194714QS~); } pool { allow members of ~ncs5508~; range 11.11.11.47 11.11.11.50; next-server 11.11.11.2; if exists user-class and option user-class = ~iPXE~ { filename=~http#//11.11.11.2#9090/ncs5500-mini-x-6.2.25.10I.iso~; } if exists user-class and option user-class = ~exr-config~ { if (substring(option vendor-class.cisco-vendor-id-vendor-class,3,11)=~FGE194714QS~) { if (substring(option vendor-class.cisco-vendor-id-vendor-class,19,99)=~NCS-5508~) { filename=~http#//11.11.11.2#9090/scripts/exhaustive_ztp_script.py~; } } } ddns-hostname ~ncs-5508-local~; option routers 11.11.11.2; }}Packet CapturesAll the relevant packet captures for this scenario have been shared on cloudshark# https#//www.cloudshark.org/captures/2b821c09ed7b and are embedded below. You can also find the corresponding .pcap file in the github repo here# https#//github.com/ios-xr/ztp-pcap/blob/master/6225/pcap/dhcpv4-filename.pcapDHCP Discover (Router to DHCP Server)Play around with the embedded cloudshark decode of the DHCP discover packet below# Interact above or View on Cloudshark   The DHCP discover packet sent by the router encodes the following information# dhcp-client-identifier (option 61)# This encodes the Serial Number of the device as a string. On the DHCP server this option can be used to identify the device during iPXE as well as during the ZTP phase based solely on the Serial Number of the device identified in the purchase order. This unique identification allows complete automation of the device bringup (image and config/script download without ever logging into the device). In the DHCP server configuration above, this match is identified in red. vendor-class-identifier (option 60) # This option encodes the following information in this example# PXEClient#Arch#00009#UNDI#003010#PID#NCS-5508 and is identical in the iPXE and ZTP phases. This can be broken down into# The type of client# e.g. PXEClient The architecture of The system (Arch)# e.g.# 00009 Identify an EFI system using a x86-64 CPU The Universal Network Driver Interface (UNDI)# e.g.# 003010 (first 3 octets identify the major version and last 3 octets identify the minor version) The Product Identifier (PID)# e.g.# NCS-5508 Vendor-Identifying Vendor Class Option (vivco/option 124)# This option can be used to enforce more granular classification of the network device and the isc-dhcp config that processes this information is marked in lime green ( ) in the dhcp server configuration above. This option was introduced to have parity with the corresponding DHCPv6 option (16) on which option 124 is modeled. This option encodes the following information# Vendor Enterprise Number# The Cisco IANA Enterprise Number is 9. This is the first 4 bytes within the option and is encoded as an integer 32 in the option. Serial Number# Option 124/vivco defines a data option field that may contain any vendor specific information (RFC 3925). The first 14 characters identifies the Serial Number# (SN# + <11 character Serial Number>). Platform PID# The remaining data option field encodes the platform name (NCS-5508 in this example). User Class Information (Option 77)# This option is used by the client (router) to identify the current stage of operation. There are two provisioning stages # IPXE# This is the image download phase and is triggered when the device is brought up without an image or is forced into the iPXE mode in the BIOS. When the network device is performing iPXE, the user-class information encodes the string# iPXE ZTP# This is the stage during which the network device expects to download the configuration/provisioning script that it should apply/execute. During this phase, the user-class information encodes the string# exr-config Don’t be alarmed by the “malformed option” indication in the packet capture embedded above for option 77. This is a known issue with wireshark due to the variation in the interpretation of option 77 ( see https#//ask.wireshark.org/questions/35332/dhcp-option-77-malformed-option/55006).DHCP Offer (DHCP Server to Router)The cloudshark decode of the DHCP offer message in response to the request described above is shown below# Interact above or View on Cloudshark   While there are multiple options that the Server responds with, there are certain options/fields in particular that the IOS-XR ZTP infrastructure utilizes to determine the location of the script/config to download and the routing required to reach the web server. These are# BOOTP Filename# While some DHCP servers might consider this obsolete (for e.g. KEA DHCP Server), there are a few legacy DHCP servers capable of setting and returning the BOOTP filename (like ISC-DHCP 4.x). When this is done (as shown by the yellow (  ) highlight), the filename gets set in the BOOTP flags ( expand the Bootp flags in the DHCP offer capture above). IOS-XR will accept this and download the target file based on it. IOS-XR will parse the BOOTP filename to identify the server IPv4 address and filename separately. Router (Option 3)# As explained above, IOS-XR uses the filename to parse the server IPv4 address and combines it with the Router IP (option 3) as the gateway to install a static route towards the server IP during the ZTP process. The remaining two captures in https#//www.cloudshark.org/captures/2b821c09ed7b identify the DHCP request/reply exchange to wrap up the complete exchange between the router and the DHCP server.DHCPv4 using Option 67 bootfile-name (IOS-XR Release 6.2.25+)Post IOS-XR release version 6.2.25, IOS-XR ZTP also supports explicitly setting the bootfile-name (option 67) in place of the BOOTP filename explained above.For this purpose we use the the DHCPv4 server configuration located here.The config is reproduced here for the reader’s convenience#option space cisco-vendor-id-vendor-class code width 1 length width 1;option vendor-class.cisco-vendor-id-vendor-class code 9 = {string};######### Network 11.11.11.0/24 ################shared-network 11-11-11-0 {####### Pools ############## subnet 11.11.11.0 netmask 255.255.255.0 { option subnet-mask 255.255.255.0;\toption broadcast-address 11.11.11.255;\toption routers 11.11.11.2;\toption domain-name-servers 11.11.11.2;\toption domain-name ~cisco.local~;\t# DDNS statements \tddns-domainname ~cisco.local.~;\t# use this domain name to update A RR (forward map) \tddns-rev-domainname ~in-addr.arpa.~; \t# use this domain name to update PTR RR (reverse map) }######## Matching Classes ########## class ~ncs5508~ { match if (substring(option dhcp-client-identifier,0,11) = ~FGE194714QS~); } pool { allow members of ~ncs5508~; range 11.11.11.47 11.11.11.50; next-server 11.11.11.2; if exists user-class and option user-class = ~iPXE~ { filename=~http#//11.11.11.2#9090/ncs5500-mini-x-6.2.25.10I.iso~; } if exists user-class and option user-class = ~exr-config~ { if (substring(option vendor-class.cisco-vendor-id-vendor-class,3,11)=~FGE194714QS~) { if (substring(option vendor-class.cisco-vendor-id-vendor-class,19,99)=~NCS-5508~) { option bootfile-name ~http#//11.11.11.2#9090/scripts/exhaustive_ztp_script.py~; } } } ddns-hostname ~ncs-5508-local~; option routers 11.11.11.2; }}Packet CapturesAll the relevant packet captures for this scenario have been shared on cloudshark# https#//www.cloudshark.org/captures/bba324c2b261 and are embedded below. You can also find the corresponding .pcap file in the github repo here# https#//github.com/ios-xr/ztp-pcap/blob/master/6225/pcap/dhcpv4-bootfilename.pcapDHCP Discover (Router to DHCP Server)Take a look at embedded cloudshark decode of the DHCP discover packet. Interact above or View on Cloudshark   The DHCP discover packet sent by the router encodes the following information# dhcp-client-identifier (option 61)# This encodes the Serial Number of the device as a string. On the DHCP server this option can be used to identify the device during iPXE as well as during the ZTP phase based solely on the Serial Number of the device identified in the purchase order. This unique identification allows complete automation of the device bringup (image and config/script download without ever logging into the device). In the DHCP server configuration above, this match is identified in red. vendor-class-identifier (option 60) # This option encodes the following information in this example# PXEClient#Arch#00009#UNDI#003010#PID#NCS-5508 and is identical in the iPXE and ZTP phases. This can be broken down into# The type of client# e.g. PXEClient The architecture of The system (Arch)# e.g.# 00009 Identify an EFI system using a x86-64 CPU The Universal Network Driver Interface (UNDI)# e.g.# 003010 (first 3 octets identify the major version and last 3 octets identify the minor version) The Product Identifier (PID)# e.g.# NCS-5508 Vendor-Identifying Vendor Class Option (vivco/option 124)# This option can be used to enforce more granular classification of the network device and the isc-dhcp config that processes this information is marked in green ( ) in the dhcp server configuration above. This option was introduced to have parity with the corresponding DHCPv6 option (16) on which option 124 is modeled. This option encodes the following information# Vendor Enterprise Number# [The Cisco IANA Enterprise Number is 9. This is the first 4 bytes within the option and is encoded as an integer 32 in the option. Serial Number# Option 124/vivco defines a data option field that may contain any vendor specific information (RFC 3925). The first 14 characters identifies the Serial Number# (SN# + <11 character Serial Number>). Platform PID# The remaining data option field encodes the platform name (NCS-5508 in this example). User Class Information (Option 77)# This option is used by the client (router) to identify the current stage of operation. There are two provisioning stages # IPXE# This is the image download phase and is triggered when the device is brought up without an image or is forced into the iPXE mode in the BIOS. When the network device is performing iPXE, the user-class information encodes the string# iPXE ZTP# This is the stage during which the network device expects to download the configuration/provisioning script that it should apply/execute. During this phase, the user-class information encodes the string# exr-config Don’t be alarmed by the “malformed option” indication in the packet capture embedded above for option 77. This is a known issue with wireshark due to the variation in the interpretation of option 77 ( see https#//ask.wireshark.org/questions/35332/dhcp-option-77-malformed-option/55006).DHCP Offer (DHCP Server to Router)The cloudshark decode of the DHCP offer message in response to the request described above is shown below# Interact above or View on Cloudshark   While there are multiple options that the Server responds with, there are certain options/fields in particular that the IOS-XR ZTP infrastructure utilizes to determine the location of the script/config to download and the routing required to reach the web server. These are# Option 67 (bootfile-name)# As part of 6.2.25, we support processing option 67 to glean the location of the script/config to download. When this option is used (as shown by the yellow (  ) highlight), the filename is NOT set in the BOOTP flags ( expand the Bootp flags in the DHCP offer capture above to see this), instead we see a separate option-67 field.. IOS-XR will accept this and download the target file based on it. It will parse the option-67 bootfile-name to identify the server IPv4 address and filename separately. Router (Option 3)# As explained above, IOS-XR uses the filename to parse the server IPv4 address and combines it with the Router IP (option 3) as the gateway to install a static route towards the server IP during the ZTP process. The remaining two captures in https#//www.cloudshark.org/captures/bba324c2b261 identify the DHCP request/reply exchange to wrap up the complete exchange between the router and the DHCP server.DHCPv6 using Option 59Post IOS-XR release version 6.2.25, IOS-XR ZTP also supports explicitly setting the bootfile-name (option 67) in place of the BOOTP filename explained above.For this purpose we use the the DHCPv4 server configuration located here.The config is reproduced here for the reader’s convenience#option dhcp6.user-class code 15 = string;option dhcp6.bootfile-url code 59 = string;option dhcp6.name-servers 2001#420#210d##a;option dhcp6.domain-search ~cisco.com~;option dhcp6.fqdn code 39 = string;option dhcp6.vendor-class code 16 = { integer 32, integer 16, string };ddns-update-style none;# option definitions common to all supported networks...option domain-name ~example.org~;option domain-name-servers ns1.example.org, ns2.example.org;default-lease-time 600;max-lease-time 7200;shared-network 2001-dba-100 { subnet6 2001#dba#100##/64 { # Range for clients range6 2001#dba#100##10 2001#dba#100##30; # Range for clients requesting a temporary address range6 2001#dba#100##/64 temporary; option dhcp6.name-servers 2001#dba#100##1; option dhcp6.domain-search ~cisco.local~; if exists dhcp6.user-class and substring(option dhcp6.user-class, 2, 4) = ~iPXE~ { option dhcp6.bootfile-url = ~http#//[2001#dba#100##1]/ncs5k-mini-4~; } else if exists dhcp6.user-class and substring(option dhcp6.user-class, 0, 10) = ~exr-config~ { if substring(option dhcp6.client-id, 1, 5) = 02#00#00#00#09 { if substring(option dhcp6.client-id, 6, 99) = 46#47#45#31#39#34#37#31#34#51#53#00 { if substring (option dhcp6.vendor-class, 43, 99) = ~NCS-5508~ { option dhcp6.bootfile-url = ~http#//[2001#dba#100##1]#9090/scripts/exhaustive_ztp_script.py~; } } } } }}Packet CapturesAll the relevant packet captures for this scenario have been shared on cloudshark# https#//www.cloudshark.org/captures/eeedef4dd779 and are embedded below. You can also find the corresponding .pcap file in the github repo here# https#//github.com/ios-xr/ztp-pcap/blob/master/6225/pcap/dhcpv6.pcapDHCPv6 SOLICIT (Router to DHCPv6 Server)Take a look at embedded cloudshark decode of the DHCPv6 Solicit packet. Interact above or View on Cloudshark   The DHCPv6 Solicit packet sent by the router encodes the following information# Client Identifier (option 1)# In DHCPv6 specification - RFC 3315 expects the client identifier to contain the DUID of the requesting device. The RFC also goes into further details on what the DUID must look like # DUID. Based off this, IOS-XR uses the following structure for the DUID field# DUID Type# There are 3 DUID types defined in the RFC above, and IOS-XR uses type 2 = “Vendor-assigned unique ID based on Enterprise Number”. This is encoded as a two type field in hex# 00#02 Enterprise Number# The Cisco IANA Enterprise Number is 9. This is the first 4 bytes within the option and as part of the hex string# 00#00#00#09This unique identification allows complete automation of the device bringup (image and config/script download without ever logging into the device). In the DHCP server configuration above, this match is identified in red. Identifier# This field contains the encoded Serial Number of the device in hex. In the above example, it is# 46#47#45#31#39#34#37#31#34#51#53#00 which translates to FGE194714QS in ascii. Thus, the resultant field is 00#02#00#00#00#09#46#47#45#31#39#34#37#31#34#51#53#00. The classification against this field is highlighted in red (  ) in the DHCPv6 server config above. vendor-class (option 16) # This option encodes the following information in this example# Enterprise-Number PXEClient#Arch#00009#UNDI#003010#PID#NCS-5508 and is identical in the iPXE and ZTP phases. This can be broken down into# Enterprise Number# The Cisco IANA Enterprise Number is 9. This is the first 4 bytes within the option and is encoded as an integer 32 in the option. The type of client# e.g. PXEClient The architecture of The system (Arch)# e.g.# 00009 Identify an EFI system using a x86-64 CPU The Universal Network Driver Interface (UNDI)# e.g.# 003010 (first 3 octets identify the major version and last 3 octets identify the minor version) The Product Identifier (PID)# e.g.# NCS-5508. If you see the above dhcpv6 server config, the part highlighted in green (  ) is used to extract and classify against the Platform ID (PID). User Class (Option 15)# This option is used by the client (router) to identify the current stage of operation. There are two provisioning stages # IPXE# This is the image download phase and is triggered when the device is brought up without an image or is forced into the iPXE mode in the BIOS. When the network device is performing iPXE, the user-class information encodes the string# iPXE ZTP# This is the stage during which the network device expects to download the configuration/provisioning script that it should apply/execute. During this phase, the user-class information encodes the string# exr-config DHCPv6 Advertise (DHCP Server to Router)The cloudshark decode of the DHCP offer message in response to the request described above is shown below# Interact above or View on Cloudshark   For DHCPv6, the primary option that the router needs from the Server is# Boot File URL (Option 59)# This Option encodes the Boot file URL that is sent back to the router in response. IOS-XR will accept this and download the target file based on it. This file may be a script or config and will be executed/applied accordingly.The remaining two captures in https#//www.cloudshark.org/captures/eeedef4dd779 identify the DHCPv6 request/reply exchange to wrap up the complete exchange between the router and the DHCPv6 server.Hopefully, this blog is useful to anyone looking to deploy ZTP with IOS-XR. Do reach out if there are any concerns and we’ll do our best to help out.", "url": "/blogs/2017-09-21-ios-xr-ztp-learning-through-packet-captures/", "author": "Akshat Sharma", "tags": "iosxr, cisco, linux, ZTP, DHCPv6, dhcp, dhcpv4, option 124, vivco, pcap, cloudshark, wireshark" } , "blogs-2018-02-22-ztp-integration-with-nso": { "title": "Using ZTP to integrate IOS-XR with NSO", "content": " Using ZTP to integrate IOS-XR with NSO Introduction NSO IOS-XR ZTP Flow of Operation DHCP Configuration ZTP Script Device Profile IntroductionNetwork Services Orchestrator is a Cisco tool that provides end-to-end orchestration that spans multiple domains in your network. Using strict, standardized YANG models for both services and devices and a highly efficient abstraction layer between your network services and the underlying infrastructure, the orchestrator lets you automate Cisco and other vendor’s devices.NSOTo automate configuration of physical devices, NSO relies on add-ons packages called Network Element Driver (NED)NSO provides 2 different ways to interact with IOS-XR# The IOS-XR CLI NED for CLI configuration The Netconf NED for Netconf/Yang configurationThe IOS-XR CLI NED must be purchased for your version of NSO. Once acquired, it should be installed inside NSO as a package.For the Netconf NED, there are 2 ways to create the package Download all the models supported by XR on github# Yang models for Cisco IOS-XR and use the ncs-make-package command to create the package, once created you can install the package inside NSO. Use the NSO pioneer tool available on github# Your Swiss army knife for NETCONF, YANG and NSO NEDs to retrieve all the models from a device and create the package. The advantage of the second method is that you are certain that the device effectively supports all the models but the retrieve operation can take some time.NSO has a set of REST/RESTCONF northbound API that can be used to provision a devices using simple HTTP GET/PUT/POST request. In short, only three operations are required# Creating the device and associate it with the correct NED (Netconf/IOS-XR CLI) Exchange the RSA keys between the device and NSO synchronize the configuration with NSOThe example in this blog uses REST, if you plan to use RESTCONF, small modifications in the URIs are required.To allow full configuration of device, NSO requires configuring the username and password (encrypted) in an authorization group, devices that belong to a specified authgroup shares the same username and password. In the example below the authgroup ios-xr-default has been created in advance on the NSO server.devices authgroups group ios-xr-default default-map remote-name cisco default-map remote-password <encrypted password>IOS-XRTo allow IOS-XR to communicate with NSO using the CLI NED, it is required to install the K9 (Crypto support) package. If you decide to use the Netconf NED, you will have to also install the MGBL (SNMP/Netconf/telemetry, etc.) in addition to the K9 package.A base configuration that enables these features should also be placed onto the device. The example described in this blog use the management interface to communicate with the NSO server, it is imperative to keep the ip address and device name constant after registering the device with NSO.ZTPZTP has support for both shell and python scripts, IOS-XR comes with an rich environment of shell tools and python libraries. In this example we will use a python based ZTP script and will leverage the python-netclient, python-json and the embedded ztp_helper libraries to provision the device in NSO.The python-netclient package provides us access to the urllib, urllib2 and base64 libraries, the python-json package allows us to manipulate json data efficiently.Flow of OperationDHCP ConfigurationFor ZTP to operate a valid IPv4/IPv6 address is required and the DHCP server must send a pointer to the configuration script via option 67. Here is an example of configuration## DHCP Server Configuration file.#allow bootp;allow booting;ddns-update-style interim;option time-offset -8;ignore client-updates;default-lease-time 300;log-facility local7;authoritative;subnet 192.168.1.0 netmask 255.255.255.0 { option subnet-mask 255.255.255.0; option broadcast-address 192.168.1.255; option routers 192.168.1.1; option domain-name-servers 192.168.1.10; option domain-name ~cisco.local~;}host ncs-5001-1 { option dhcp-client-identifier ~FOC2018R1QU~; fixed-address 192.168.1.16; if exists user-class and option user-class = ~iPXE~ { filename = ~http#//192.168.2.10/ipxe/ncs5k.ipxe~; } elsif exists user-class and option user-class = ~exr-config~ { filename = ~http#//192.168.2.10/scripts/ncs-5001_nso_ztp.py~; }}ZTP ScriptThe ZTP script will do the following operations# Install the K9SEC and MGBL packages. Create a general-purpose key. Apply a basic configuration that allow the netconf agent to communicate with the NSO server. Push the Device profile to NSO Push the RSA key to NSO Synchronize the basic configuration with NSO Device Profile The device profile is described in a JSON template with the name and ip address of the device filled during the ZTP execution, the IOS-XR CLI based template looks like this# myDevice = { ~device~# { ~name~# ~XXXX~, ~address~# ~XXXX~, ~port~# 22, ~state~# { ~admin-state~# ~unlocked~ }, ~authgroup~# ~ios-xr-default~, ~device-type~# { ~cli~# { ~ned-id~# ~tailf-ned-cisco-ios-xr-id#cisco-ios-xr~ } } }}If you plan to use Netconf, the profile template will have to be modified#myDevice = { ~device~# { ~name~# ~XXXX~, ~address~# ~XXXX~, ~port~# 22, ~state~# { ~admin-state~# ~unlocked~ }, ~authgroup~# ~ios-xr-default~, ~device-type~# { ~netconf~# { ~ned-id~# ~tailf-ncs-ned#netconf~ } } }}A simple function has been created that can handle all REST operation with NSOdef nso_rest(self, ressource=None, operation=None, data=None)# ~~~ ~~~ if data is not None# request = urllib2.Request(ressource, data) else# request = urllib2.Request(ressource) request.add_header(~Authorization~, ~Basic %s~ % base64.encodestring('%s#%s' % (NSO_USER, NSO_PASSWD)).replace('\\n', '')) request.add_header('Content-Type', 'application/vnd.yang.data+json') request.get_method = lambda# operation try# response = urllib2.urlopen(request) except urllib2.HTTPError,error# self.syslogger.info(~REST operation error code# ~ + str(error.code())) self.syslogger.info(~REST response# ~ + error.read()) except URLError, error# self.syslogger.info(~URL error# ~ + str(error.code())) else# self.syslogger.info(~REST operation status# ~ + str(response.code)) self.syslogger.info(~REST return data# ~ + response.read()) return {~status~# response.code, ~output~ # response.read()}This function will be called like thisztp_script.syslogger.info(~###### Pushing Device Profile ######~)ztp_script.nso_rest(BASE_URI + '/devices/device/' + hostname, 'PUT', json.dumps(myDevice))ztp_script.syslogger.info(~###### Pushing RSA key ######~)ztp_script.nso_rest(BASE_URI + '/devices/device/' + hostname + '/ssh/_operations/fetch-host-keys/', 'POST')ztp_script.syslogger.info(~###### Syncing configuration ######~)ztp_script.nso_rest(BASE_URI + '/devices/device/' + hostname + '/_operations/sync-from', 'POST')The complete detail of the ZTP script is available on github integrating IOS-XR with NSO", "url": "/blogs/2018-02-22-ztp-integration-with-nso/", "author": "Patrick Warichet", "tags": "iosxr, Orchestration, NSO" } , "tutorials-2016-07-27-ipxe-deep-dive": { "title": "iPXE Deep Dive", "content": " IOS-XR# iPXE Deep Dive Introduction Topology Boot Process iPXE DHCPv4 Request iPXE DHCPv6 Request iPXE without Chainloading DHCP Server Configuration DHCPv4 DHCPv6 Dynamic Scripting - Embedding iPXE variables in URL iPXE with Chainloading iPXE Script Conclusions IntroductioniPXE is an open source boot firmware (licensed under the GNU GPL with some portions under GPL-compatible licenses). It is fully backward compatible with PXE but include several enhancement. The enhancement that are important for IOS-XR are the following# Boot from a web server via HTTP control the boot process with scripts control the boot process with menus DNS supportiPXE is included in the network card of the management interfaces only and support for iPXE boot is included in the system firmware (UEFI) of the NCS1K, NCS5k and NCS5500 series routers. All these systems are equipped with a UEFI 64-bits firmware (aka BIOS).iPXE can run on both IPv4 and IPv6 protocol but cannot use SLAAC for IPv6.iPXE enumerates all Ethernet interfaces net0, net1, net2, …TopologyIn the following examples we will use a NCS-5001 router. This device is equipped with 2 management interfaces but we will use only one of these two interfaces, It is recommended to place each interfaces in different subnet to facilitate the management process and improve redundancy.The following diagram show the topology used for all examples. Both the DHCP and HTTP server are on a different subnet than the NCS-5001.Boot ProcessThe IOS-XR 6.0 boot process is illustrated below, iPXE requires two external services, a DHCP server (e.g. isc-dhcpd) and a HTTP server (e.g. Apache)It is important to note that a different dhcp client will start at the end of the boot process. This second dhcp client will facilitate auto provisioning the system.By default all NCS series router boot from the local disk, there are 2 options to force the system to boot using iPXE# If the device is already booted you can issue the command “hw-module location bootmedia network reload~ in admin mode to force the system to reboot in iPXE mode.RP/0/RP0/CPU0#ios#adminSun Apr 10 00#56#08.037 UTCroot connected from 127.0.0.1 using console on xr-vm_node0_RP0_CPU0sysadmin-vm#0_RP0# hw-module location all bootmedia network reloadSun Apr 10 00#56#24.167 UTCReload hardware module ? [no,yes] yesresult Card reload request on all succeeded.If the system is just being powered on, you can get to the device firmware by pressing <ESC> or <DEL> after it has completed the hardware diagnostic. You will be presented with the Boot selection menu. To force the device to boot using iPXE select the first entry “UEFI# Built-in EFI IPXE”Once the option is selected, iPXE will initialize the management interfaces, display the features options that were included in the iPXE firmware and propose you to jump into the iPXE prompt by pressing <CTRL>BiPXE initialising devices...okiPXE 1.0.0+ (aa070) -- Open Source Network Boot Firmware -- http#//ipxe.orgFeatures# DNS HTTP TFTP VLAN EFI ISO9660 NBI MenuPress Ctrl-B to drop to iPXE shelliPXE>When presented with the iPXE command line, you are in the iPXE environment, there are multiple commands that can be used for manual booting and for diagnosing problems. Commands can also be used as part of an iPXE script (see section iPXE with chain loading).You can use the help command to show a list of all available commands. Full documentation for each command is provided in the iPXE command reference http#//ipxe.org/cmd.iPXE DHCPv4 RequestAfter Initializing the management Interface Ethernet driver, iPXE will send a DHCP request, DHCP will send both an IPv6 and an IPv4 request, The capture below show the initial IPv4 DHCP request sent by the system. Bootstrap Protocol Message type# Boot Request (1) Hardware type# Ethernet (0x01) Hardware address length# 6 Hops# 0 Transaction ID# 0x20e69f64 Seconds elapsed# 4 Bootp flags# 0x8000 (Broadcast) 1... .... .... .... = Broadcast flag# Broadcast .000 0000 0000 0000 = Reserved flags# 0x0000 Client IP address# 0.0.0.0 (0.0.0.0) Your (client) IP address# 0.0.0.0 (0.0.0.0) Next server IP address# 0.0.0.0 (0.0.0.0) Relay agent IP address# 0.0.0.0 (0.0.0.0) Client MAC address# 52#46#27#70#1a#67 (52#46#27#70#1a#67) Client hardware address padding# 00000000000000000000 Server host name not given Boot file name not given Magic cookie# DHCP Option# (53) DHCP Message Type Length# 1 DHCP# Discover (1) Option# (57) Maximum DHCP Message Size Length# 2 Maximum DHCP Message Size# 1472 Option# (93) Client System Architecture Length# 2 Client System Architecture# EFI x86-64 (9) Option# (94) Client Network Device Interface Length# 3 Major Version# 3 Minor Version# 10 Option# (60) Vendor class identifier Length# 45 Vendor class identifier# PXEClient#Arch#00009#UNDI#003010#PID#NCS-5001 Option# (77) User Class Information Length# 4 User Class identifier# iPXE Option# (55) Parameter Request List Length# 22 Parameter Request List Item# (1) Subnet Mask Parameter Request List Item# (3) Router Parameter Request List Item# (6) Domain Name Server Parameter Request List Item# (7) Log Server Parameter Request List Item# (12) Host Name Parameter Request List Item# (15) Domain Name Parameter Request List Item# (17) Root Path Parameter Request List Item# (43) Vendor-Specific Information Parameter Request List Item# (60) Vendor class identifier Parameter Request List Item# (66) TFTP Server Name Parameter Request List Item# (67) Bootfile name Parameter Request List Item# (119) Domain Search Parameter Request List Item# (128) PXE - undefined (vendor specific) Parameter Request List Item# (129) PXE - undefined (vendor specific) Parameter Request List Item# (130) PXE - undefined (vendor specific) Parameter Request List Item# (131) PXE - undefined (vendor specific) Parameter Request List Item# (132) PXE - undefined (vendor specific) Parameter Request List Item# (133) PXE - undefined (vendor specific) Parameter Request List Item# (134) PXE - undefined (vendor specific) Parameter Request List Item# (135) PXE - undefined (vendor specific) Parameter Request List Item# (175) Etherboot Parameter Request List Item# (203) Unassigned Option# (175) Etherboot Length# 36 Value#2969895296,2248423659,50397184,385941796,16847617,19529985,654377248, 16848129,19267841 Option# (61) Client identifier Length# 11 Hardware type# DUID (0xFF) Client Identifier# 46#4f#43#31#39#34#37#52#31#34#33 (FOC1947R143) Option# (97) UUID/GUID-based Client Identifier Length# 17 Client Identifier (UUID)# 9ed138b2-dc55-42b6-9c56-2cf6f63921d9 Option# (255) End Option End# 255iPXE includes a number of options in the initial IPv4 DHCP request, the relevant ones are highlightedOption 60# “vendor-class-identifier” Identify 4 elements separated by columns#1 The type of client# e.g.# PXEClient2 The architecture of The system (Arch)# e.g.# 00009 Identify an EFI system using a x86-64 CPU3 The Universal Network Driver Interface (UNDI)# e.g.# 003010 (first 3 octets identify the major version and last 3 octets identify the minor version)4 The Product Identifier (PID)# e.g.# NCS-5001Option 61# “dhcp-client-identifier” Identify the Serial Number of the systemOption 66 and 67# are used for TFTP, the first one request the TFTP server name while the second request the filenameOption 77# “user-class” Identify the mode of the system# e.g.# iPXEOption 97# “uuid” Identify the Universally Unique Identifier a 128-bit value (not usable at this time)Option 128 - 135# Reserved for PXE boot variables but not in use.In is response the DHCP server will place the bootfile URI in option 67 “filename” or “bootfile-name” e.g.#http#//172.30.0.22/ncs5k/6.0.0/ncs5k-mini-x.iso-6.0.0iPXE DHCPv6 Request DHCPv6 Message type# Relay-forw (12) Hopcount# 0 Link address# fd#30#12##1 (fd#30#12##1) Peer address# fe80##c672#95ff#fea7#efc0 (fe80##c672#95ff#fea7#efc0) Relay Message Option# Relay Message (9) Length# 88 Value# 011ac36d00010012000200000009464f4331393437523134... DHCPv6 Message type# Solicit (1) Transaction ID# 0x1ac36d Client Identifier Option# Client Identifier (1) Length# 18 Value# 000200000009464f43313934375231343300 DUID# 000200000009464f43313934375231343300 DUID Type# assigned by vendor based on Enterprise number (2) Enterprise ID# ciscoSystems (9) Identifier# 464f43313934375231343300 (FOC1947R143) Identity Association for Non-temporary Address Option# Identity Association for Non-temporary Address (3) Length# 12 Value# 1d4098ed0000000000000000 IAID# 1d4098ed T1# 0 T2# 0 Option Request Option# Option Request (6) Length# 8 Value# 00170018003b003c Requested Option code# DNS recursive name server (23) Requested Option code# Domain Search List (24) Requested Option code# Bootfile URL(59) Requested Option code# Bootfile Prameters (60) User Class Option# User Class (15) Length# 6 Value# 000469505845 # ~iPXE~ Vendor Class Option# Vendor Class (16) Length# 14 Value# 0000000900084e43532d35303031 Enterprise ID# ciscoSystems (9) vendor-class-data# NCS-5001 Elapsed time Option# Elapsed time (8) Length# 2 Value# 0000 Elapsed-time# 0 ms Interface-Id Option# Interface-Id (18) Length# 4 Value# 0000001d Interface-ID# The initial DHCPv6 solicit has the relevant option highlightedOption 1# “client-identifier” equivalent to DHCPv4 option 61 but with the following format#DUID Type# integer 16 e.g.# 0002 (assigned by vendor)Enterprise Id# integer 32 e.g.# 00000009 (Cisco Systems)Client Identifier# string e.g.# FOC1947R143Option 15# “dhcp6.user-class” equivalent to DHCPv4 option 77 but the first 2 Octets define the length of the stringOption 16# “vendor-class-identifier” equivalent to DHCPv4 option 60 but with the following format#Enterprise Id# integer 32 e.g.#00000009 (Cisco Systems)Length# integer 16Vendor# string e.g.# NCS-5001Option 59# “dhcp6.bootfile-url” equivalent to DHCPv4 option 67Option 60# “dhcp6.bootfile-parameter” required to be present but not in use.The DHCPv6 server will include option 59 “dhcp6.bootfile-url” in its response containing the full URL of the boot image e.g.# [http#//[fd#30##172#30#0#22]/ncs5k/6.0.0/ncs5k-mini-x.iso-6.0.0] (http#//[fd#30##172#30#0#22]/ncs5k/6.0.0/ncs5k-mini-x.iso-6.0.0)iPXE without ChainloadingIn this first examples, iPXE features are not used but the usage is similar to PXE boot. In the following examples we will rely solely on the DHCP server configuration to provide the elements necessary to identify the boot ISO for the device.DHCP Server ConfigurationDHCPv4Using the options above we can configure isc-dhcpd to adequately provide the URI to boot the system, the common statements for the network and the pool are shown below########## Network 172.30.12.0/24 ################shared-network 172-30-12-0 { subnet 172.30.12.0 netmask 255.255.255.0 { option subnet-mask 255.255.255.0; option broadcast-address 172.30.12.255; option routers 172.30.12.1; option domain-name-servers 172.30.0.25; option domain-name ~cisco.local~; } ####### Pool ######### pool { range 172.30.12.10 172.30.12.100; next-server 172.30.0.22; if exists user-class and option user-class = ~iPXE~ { filename = ~http#//172.30.0.22/ncs5k-mini-4~; } else if exists user-class and option user-class = ~exr-config~ { filename = ~http#//172.30.0.22/scripts/ncs-ztp.sh~; }In the example above option 77 is used to provide the bootfile to the system, the if-then-else statement is required to prevent the DHCP server to provide the (large) bootfile to the auto-configuration process. With this configuration all system in iPXE mode will receive a DHCP offer with identical bootfile URI.If we want to add more granularity to the process we can define a class and using option 60 to only target a specific product or a family of products using the PID embedded in the the request. in the match statement we first verify that the system is in iPXE mode by matching the beginning of the vendor-class-identifier “PXEClient” than we match the first 6 octets of the PID portion “NCS-50”, this will match all the NCS-5K routers (NCS-5001, NCS-5002, NCS-5011) and provide them the same bootfile URI, the “if” statement can be more specific to only match NCS-5001 product and additional “else-if” statement can be added to match other products.######### Class ######### class ~ncs-5k~ { match if substring (option vendor-class-identifier, 0, 9) = ~PXEClient~; if substring (option vendor-class-identifier, 37, 6) = ~NCS-50~ { filename = ~http#//172.30.0.22/ncs5k-mini-3~; } }Granularity of the boot image can be even more specific, the traditional approach is to use the mac address with a host definition inside the pool, as illustrated below######## Hosts #########host ncs-5001-a { hardware ethernet c4#72#95#a7#ef#c2; if exists user-class and option user-class = ~iPXE~ { filename = ~http#//172.30.0.22/ncs5k-mini-1~; } fixed-address 172.30.12.50;}Using the host statement we provide a fixed address which can be useful for DNS, we still need to verify that option 77 is set to iPXE in the request to only provide the bootfile when required. The disadvantage of using the mac-address is that it is not necessary know in advance and is not written on the packaging box if this is the initial bootup of the system, another approach would be to use the uuid (option 97) or the serial number embedded in option 61.######## Hosts #########host ncs-5001-b { option dhcp-client-identifier ~FOC1947R144~; if exists user-class and option user-class = ~iPXE~ { filename = ~http#//172.30.0.22/ncs5k-mini-2~; } fixed-address 172.30.12.52;}Using the different options and the flexibility of the ISC dhcp server we can achieve various degrees of granularity for the system we want to iPXE boot, but using DHCP options does not scale and each change require to restart the DHCP service. It offers the advantage to be identical to PXE and is easy to do for small to medium size network.DHCPv6The ISC-DHCP service is mono stack, to support IPv6 a second instance of the service needs to be launched, both instances should use different configuration file. The common configuration statement for the DHCPv6 is as follow#shared-network FD-30-12 { subnet6 fd#30#12##/64 { # Range for clients range6 fd#30#12##1024 fd#30#12##1124; # Range for clients requesting a temporary address range6 fd#30#12##/64 temporary; # Additional options option dhcp6.name-servers fd#30##172#30#0#25; option dhcp6.domain-search ~cisco.local~; if exists dhcp6.user-class and substring(option dhcp6.user-class, 2, 4) = ~iPXE~ { option dhcp6.bootfile-url = ~http#//[fd#30##172#30#0#22]/ncs5k-mini-4~; } else if exists dhcp6.user-class and substring(option dhcp6.user-class, 0, 10) = ~exr-config~ { option dhcp6.bootfile-url = ~http#//[fd#30##172#30#0#22]/scripts/ncs-ztp.sh~; } }}The DHCP configuration for IPv6 is similar to IPv4, the first 2 octets of the user-class define the length of the string, so we need to use the substring() statement to match “iPXE”. Another difference is the square brackets used to represent the IPv6 address in isc-dhcp configuration file. iPXE cannot used SLAAC and you need to disable SLAAC on the first hop router and force statefull IPv6 address assignment on the segment. For reference here is a snippet of an Cisco IOS configuration. since the DHCP and HTTP server are on a different subnet a helper-address and a relay address have been configured for DHCPv4 and DHCPv6.interface GigabitEthernet2/0 description ** Management Network ** ip dhcp relay information trusted ip address 172.30.12.1 255.255.255.0 ip helper-address 172.30.0.25 ip virtual-reassembly in ipv6 address FD#30#12##1/64 ipv6 nd managed-config-flag ipv6 nd other-config-flag ipv6 nd router-preference High ipv6 dhcp relay destination FD#30##172#30#0#25 GigabitEthernet1/0endIf there are no router present on the segment, you will have to launch the Router Advertisement Daemon (radvd) and force IPv6 routing on the DHCP server. An example of radvd.conf is as follow#interface eth1{ MinRtrAdvInterval 5; MaxRtrAdvInterval 60; AdvSendAdvert on; AdvOtherConfigFlag on; IgnoreIfMissing off; prefix FD#30#12##/64 { };};Granularity in identifying the boot image is similar to IPv4, A class that encompass all NCS-5K series can be defined as follow (ipv6 class support is available starting isc-dhcp-server 4.3.4)########## Class #########class ~ncs-5k~ { match if exists vendor-class-identifier and substring(vendor-class-identifier, 6, 6) = ~NCS-50~; if exists dhcp6.user-class and substring(option dhcp6.user-class, 2, 4) = ~iPXE~ { filename = ~http#//[fd#30##172.30.0.22]/ncs5k-mini-3~; }}Granularity to the host level can be achieve by using the serial number as identifier since the client sends the serial number as part of the client-id a simple solution is to match the complete hex data of the option.######## Hosts #########host ncs-5001-b { host-identifier option dhcp6.client-id 00#02#00#00#00#09#46#4f#43#31#39#34#37#52#31#34#33#00; if exists dhcp6.user-class and substring(option dhcp6.user-class, 2, 4) = ~iPXE~ { option dhcp6.bootfile-url = ~http#//[fd#30##172#30#0#22]/ncs5k-mini-2~; } fixed-address6 fd#30#12##172.30.12.52;}Refer to the section iPXE DHCPv6 Request on how to decode the dhcp6.client-id or use xxd to translate it in ascii.cisco@galaxy-42$ echo ~00#02#00#00#00#09#46#4f#43#31#39#34#37#52#31#34#33~ | xxd -pe -r && echo -e FOC1947R143cisco@galaxy-42$ echo -n ~FOC1947R143~ | od -A n -t x1 | sed 's/^ /00#02#00#00#00#09#/' | sed 's/ /#/g' 00#02#00#00#00#09#46#4f#43#31#39#34#37#52#31#34#33Dynamic Scripting - Embedding iPXE variables in URLThe URL provided by the DHCP server does not have to be a static. For example, you could direct iPXE to boot from the URLhttp#//172.30.0.22/boot.php?mac=${net0/mac}&product=${product#uristring}&serial=${serial#uristring}Which would expand to a URL such ashttp#//172.30.0.22/boot.php?mac=c4#72#95#a7#ef#c0&product=NCS5001&serial=FOC1947R143The boot.php program running on the web server could dynamically generate a script based on the information provided in the URL. For example, boot.php could look up the serial number in a MySQL database to determine the correct target to boot from, and then dynamically generate a script such as<?php header ( ~Content-type# text/plain~ ); echo ~#!ipxe \\n~; echo ~set myURL http#//172.30.0.22/Cisco/NCS/NCS5001/FOC1947R143 \\n~; echo ~boot myURL \\n~;?>iPXE with ChainloadingChainloading is the capability to jump from one boot statement to another. Using chainloading and the embedded scripting capability of iPXE we can have a very detail and complex selection mechanism for the boot image. In the following example we will use the boot file structure illustrated below and we will use the initial DHCP configuration described earlier but in place of providing the URI for an ISO the DHCP server will provide the URI to a iPXE boot script (boot.ipxe).The file boot.ipxe file is a script that will identify the correct image based on available iPXE variable, it starts with the “!ipxe” statement and include statement like chain isset, etc.. All the iPXE statements are documented in the iPXE command section open source boot firmwareThe script is evaluated top to bottom and works for both IPv4 and IPv6iPXE Script!ipxe # Global variables used by all other iPXE scriptschain --autofree boot.ipxe.cfg || # Boot <boot-url>/<boot-dir>/hostname-<hostname>.ipxe# if hostname DHCP variable is set and script is presentisset ${hostname} && chain --replace --autofree ${boot-dir}hostname-${hostname}.ipxe || # Boot <boot-url>/<boot-dir>/uuid-<UUID>.ipxe# if SMBIOS UUID variable is set and script is present (not usable see CSCuz28164)isset ${uuid} && chain --replace --autofree ${boot-dir}uuid-${uuid}.ipxe || # Boot <boot-url>/<boot-dir>/mac-010203040506.ipxe if script is presentchain --replace --autofree ${boot-dir}mac-${mac#hexraw}.ipxe || # Boot <boot-url>/<boot-dir>/serial-FOC1947R143.ipxe if script is presentisset ${serial} && chain --replace --autofree ${boot-dir}serial-${serial}.ipxe || # Boot <boot-url>/<boot-dir>/pid-<product>.ipxe if script is presentisset ${product} && chain --replace --autofree ${boot-dir}pid-${product}.ipxe ||# Boot <boot-url>/menu.ipxe script if all other options have been exhaustedchain --replace --autofree ${menu-url} ||chain --replace --autofree ${menu-url6} ||The first action of the script is to import a set of variables from boot.ipxe.cfg this will set ${boot-url} / ${boot-url6} and other variables.The script verify if a specific variable has been set either in the SMBIOS of the system or in the DHCP response from the server.If the variable has been set, the script attempts to jump to a secondary boot file. For example if the serial number is set “isset ${serial}”, the script will attempt to jump to the file /serial-FOC1947R144.ipxe if the file exists. If the file exist iPXE will start executing statement from this boot script. If the file does not exist the script continue to the next statement until it reaches the menu statement, the last statement of the list.Here is an example of a secondary boot script based on the serial number of the device, as you can see this script points to the last element of the chain# the ISO boot file.cisco@galaxy-42#/var/www/html/ipxe$ cat serial-FOC1947R143.ipxe#!ipxeechoecho Booting NCS5K Mini ISO 6.0.0 from ISO for ${initiator}chain --replace --autofree ${boot-url}ncs5k-mini-x.iso-6.0.0 ||chain --replace --autofree ${boot-url6}ncs5k-mini-x.iso-6.0.0Finally if all boot items have failed, the menu.ipxe script is executed and propose an interactive menu-driven list of boot options.Below is the example script for the boot menu, this example is adapted from https#//gist.github.com/robinsmidsrod/2234639Each menu items can be associated with a shortcut key and navigation between items is done using the up and down arrows, for xrv9k image we have to use the sanboot option, for NCS-5K and NCS-5500 device we use the boot keyword.boot.ipxe.cfg#!ipxe# Base URL used to resolve most resources# Should always end with a slashset boot-url http#//172.30.0.22/set boot-url6 http#//[fd#30##172#30#0#22]/# What URL to use when sanbooting# Should always end with a slashset sanboot-url http#//172.30.0.22/set sanboot-url6 http#//[fd#30##172#30#0#22]/# Relative directory to boot.ipxe used to# override boot script for specific clientsset boot-dir ipxe/# Absolute URL to the menu script, used by boot.ipxe# and commonly used at the end of simple override scripts# in ${boot-dir}.set menu-url ${boot-url}menu.ipxeset menu-url6 ${boot-url6}menu.ipxeset initiator ${product} - ${serial}boot.ipxe!ipxe# Variables are specified in boot.ipxe.cfg# Some menu defaultsset menu-timeout 30000set submenu-timeout ${menu-timeout}isset ${menu-default} || set menu-default exit###################### MAIN MENU #####################################startmenu iPXE boot menu for ${initiator}item --gap -- ------------------------- XRV9K Boot Menu ------------------------------item --key a sunstone-mini Boot xrv9k Mini 6.0.0 ISOitem --key d sunstone-latest Boot xrv9k Mini 6.1.1 ISO Latest builditem --key e sunstone-disk Boot xrv9k from local diskitem --gap -- ------------------------ NCS5000 Boot Menu -----------------------------item --key f ncs5000-6.0.0 Boot ncs-5000 Mini 6.0.0 ISOitem --key g ncs5000-6.1.1 Boot ncs-5000 Mini 6.1.1 ISOitem --gap -- ------------------------ NCS5500 Boot Menu -----------------------------item --key h ncs5500-6.0.0 Boot ncs-5500 Mini 6.0.0 ISOitem --key i ncs5500-6.1.1 Boot ncs-5500 Mini 6.1.1. Latest ISOitem --gap -- ------------------------- Advanced options -----------------------------item --key j config Configure settingsitem shell Drop to iPXE shellitem reboot Reboot Systemitemitem --key x exit Exit iPXE and continue BIOS bootchoose --timeout ${menu-timeout} --default ${menu-default} selected || goto cancelset menu-timeout 0goto ${selected} #cancelecho You cancelled the menu, dropping you to a shell #shellecho Type 'exit' to get the back to the menushellset menu-timeout 0set submenu-timeout 0goto start #failedecho Booting failed, dropping to shellgoto shell #rebootreboot #exitexit #configconfiggoto start #backset submenu-timeout 0clear submenu-defaultgoto start ############ MAIN MENU ITEMS ############ #sunstone-miniecho Booting XRV9K Mini 6.0.0 from ISO for ${initiator}sanboot ${sanboot-url}xrv9k-mini-x.iso-6.0.0 ||sanboot ${sanboot-url6}xrv9k-mini-x.iso-6.0.0 || goto failedgoto start #sunstone-latestecho Booting XRV9K Mini 6.1.1 latest developer release from ISO for ${initiator}sanboot ${sanboot-url}xrv9k-mini-latest.iso ||sanboot ${sanboot-url6}xrv9k-mini-latest.iso || goto failedgoto start #ncs5000-6.0.0echoecho Booting NCS-5K Mini ISO 6.0.0 from ISO for ${initiator}boot ${boot-url}ncs5000-mini.official ||boot ${boot-url6}ncs5000-mini.official || goto failedgoto start #ncs5000-6.1.1echoecho Booting NCS-5K Mini ISO 6.1.1 from ISO for ${initiator}boot ${boot-url}ncs5000-mini.latest ||boot ${boot-url6}ncs5000-mini.latest || goto failedgoto start #ncs5500-6.0.0echoecho Booting NCS-5500 Mini ISO 6.0.0 from ISO for ${initiator}boot ${boot-url}ncs5500-mini.official ||boot ${boot-url6}ncs5500-mini.official || goto failedgoto start #ncs5500-6.1.1echoecho Booting NCS-5500 Mini ISO 6.1.1 from ISO for ${initiator}boot ${boot-url}ncs5500-mini.latest ||boot ${boot-url6}ncs5500-mini.latest || goto failedgoto start #sunstone-diskecho Start XRV9K from disksanboot --no-describe --drive 0x80 || goto failed goto startHere is a screenshots of the boot process with only the DHCPv6 service active and no valid boot file present.iPXE> autoboot net0 <- autoboot from the mgmt interface net0# c4#72#95#a7#ef#c0 using dh8900cc on PCI01#00.1 (open) [Link#up, TX#108 TXE#0 RX#5188624 RXE#5186887]Configuring (net0 c4#72#95#a7#ef#c0).......... oknet0# fe80##c672#95ff#fea7#efc0/64net0# fd#30#12##1124/64 gw fe80##fa72#eaff#fe8b#ce80 <- ipv6 statefull address assignment Filename# http#//[fd#30##172#30#0#22]/boot.ipxe <- ipv6 boot URI from DHCPv6http#//[fd#30##172#30#0#22]/boot.ipxe... ok <- boot script is downloaded /boot.ipxe.cfg... ok <- boot variable are chained/ipxe/uuid-03000200-0400-0500-0006-000700080009.ipxe... No such file or directory (http#//ipxe.org/2d0c618e)/ipxe/mac-c47295a7efc0.ipxe... No such file or directory (http#//ipxe.org/2d0c618e)/ipxe/serial-FOC1947R143.ipxe... No such file or directory (http#//ipxe.org/2d0c618e)/ipxe/pid-NCS-5001.ipxe... No such file or directory (http#//ipxe.org/2d0c618e)http#//172.30.0.22/menu.ipxe... Network unreachable (http#//ipxe.org/280a6090)http#//[fd#30##172#30#0#22]/menu.ipxe... ok <- boot menu is executed iPXE boot menu for NCS-5001 - FOC1947R143 ------------------------- XRV9K Boot Menu ------------------------------ Boot xrv9k Mini 6.0.0 ISOBoot xrv9k Mini 6.1.1 ISO Latest buildBoot xrv9k from local disk------------------------ NCS5000 Boot Menu -----------------------------Boot ncs-5000 Mini 6.0.0 ISOBoot ncs-5000 Mini 6.1.1 ISO Latest build------------------------ NCS5500 Boot Menu -----------------------------Boot ncs-5500 Mini 6.0.0 ISOBoot ncs-5500 Mini 6.1.1 Latest build------------------------- Advanced options -----------------------------Configure settingsDrop to iPXE shellReboot System Exit iPXE and continue BIOS bootIf we select the entry “Boot ncs-5000 Mini 6.0.0 ISO”, the script will first attempt to boot using the IPv4 address, since our device did not receive a valid IPv4 address it will attempt to use the IPv6 address and start the NOS installation.Booting Skywarp Mini ISO 6.0.0 from ISO for NCS-5001 - FOC1947R143http#//172.30.0.22/ncs5000-mini.official... Network unreachable (http#//ipxe.org/280a6090)http#//[fd#30##172#30#0#22]/ncs5000-mini.official... okBooting iso-image@0x42e2cb000(835930112), bzImage@0x42e2f7000(4473806)ConclusionsiPXE offers a wide variety of configuration paradigm that can be used in large deployment, with its scripting capability, iPXE is independent of DHCP configuration and can achieve very good granularity based on model number, serial number, mac-address, host name, etc.Creation of boot file can be automated easily on the back-end side without restarting any services. On the HTTP server symbolic link can be used to move devices from one ISO to another without reconfiguration. with its backward compatibility with PXE and its low resources requirement, it is a very good alternative to ONIE.Future enhancement to the boot process including secure boot will bring even more security to the iPXE without using HTTPS.", "url": "/tutorials/2016-07-27-ipxe-deep-dive/", "author": "Patrick Warichet", "tags": "iosxr, cisco, iPXE" } , "tutorials-2016-08-06-introduction-to-rpm": { "title": "IOS-XR and RPM Package Manager", "content": " IOS-XR and RPM package manager Introduction XR Packages Installation XR Package Structure Packages Installation from the Shell Analyzing Package Using Linux IntroductionWith IOS XR 6.0, the Package Installation Envelope (PIE) format has been discarded in favor of the RPM Package Manager (RPM) format. This move aligns IOS XR 6.0 and above more closely with RPM-based Linux distribution like Red Hat or Centos. RPM is a free software project and is released under GPL. Briefly, RPMs contain the following elements# CPIO archive# Contains the absolute path of all the package’s files; Metadata# Contains the package dependencies; Scriptlets# Scripts that perform pre and post (un)installation tasks;The Metadata is a XML file that helps determine and resolve package dependencies, to facilitate dependencies resolution the metadata is used to poulate a small database located in /var/lib/rpm that can be queried using tools like YUM or RPM.A Software Maintenance Update (SMU) will be published in the form of tape archive (tar) format. The SMU will contain the following files# A Readme.txt file describing the content of the SMU; One or more RPMs; A Package-mdata.txt that contains a MD5 checksum of all the packages in the tar file;XR Packages InstallationWithin IOS-XR, two new CLI commands have been introduced that complement the existing ones# “install update” and “install upgrade”, described Table 1. These new commands require an external packages repository accessible through FTP/SFTP/SCP/TFTP or HTTP. Command Description install update source When no package is specified, update latest SMUs of all installed packages. install upgrade source version Upgrade the base image to the specified version. All installed packages are upgraded to same release as the base package. RP/0/RP0/CPU0#pwa-rtr#install update source ? WORD Enter source directory for the package(s) Example# sftp#//user@server/directory/ scp#//user@server/directory/ ftp#//user@server/directory/ tftp#//server/directory/ http#//server/directory/In the example below the k9sec package is installed using the “install update” command. After initiating the command, you can issue a “show install request” to monitor the status of the package installation.RP/0/RP0/CPU0#pwa-rtr#install update source http#//192.168.122.1#8080/xrv9k xrv9k-k9secSat Feb 13 17#18#59.981 UTC++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Update in progress...Scheme # httpHostname # 192.168.122.1#8080Collecting software state..Update packages # xrv9k-k9secFetching .... xrv9k-k9sec-1.0.0.0-r600.x86_64.rpm-6.0.0Adding packages xrv9k-k9sec-1.0.0.0-r600.x86_64.rpm-6.0.0Feb 13 17#19#08 Install operation 22 started by root#install add source /misc/disk1/install_tmp_staging_area/6.0.0 xrv9k-k9sec-1.0.0.0-r600.x86_64.rpm-6.0.0Feb 13 17#19#09 Install operation will continue in the backgroundFeb 13 17#19#12 Install operation 20 finished successfullyInstall add operation successfulActivating xrv9k-k9sec-1.0.0.0-r600Feb 13 17#19#14 Install operation 23 started by root# install activate pkg xrv9k-k9sec-1.0.0.0-r600Feb 13 17#19#14 Package list#Feb 13 17#19#14 xrv9k-k9sec-1.0.0.0-r600Feb 13 17#19#16 Install operation will continue in the background RP/0/RP0/CPU0#pwa-rtr# This product contains cryptographic features and is subject to UnitedStates and local country laws governing import, export, transfer anduse. Delivery of Cisco cryptographic products does not imply third-partyauthority to import, export, distribute or use encryption. Importers,exporters, distributors and users are responsible for compliance withU.S. and local country laws. By using this product you agree to complywith applicable laws and regulations. If you are unable to comply withU.S. and local laws, return this product immediately.A summary of U.S. laws governing Cisco cryptographic products may befound at#http#//www.cisco.com/wwl/export/crypto/tool/stqrg.htmlIf you require further assistance please contact us by sending email toexport@cisco.com.Feb 13 17#20#12 Install operation 21 finished successfullyAll the install commands log their progress. You can enter “show install log” to review the installation log file. This command is useful for identifying the reason for any failure.RP/0/RP0/CPU0#pwa-rtr#show install log 21Tue Feb 23 17#54#00.961 UTCFeb 23 17#51#14 Install operation 21 started by root# install activate pkg xrv9k-k9sec-1.0.0.0-r600Feb 23 17#51#14 Package list#Feb 23 17#51#14 xrv9k-k9sec-1.0.0.0-r600Feb 23 17#51#15 Action 1# install prepare action startedFeb 23 17#51#17 Install operation will continue in the backgroundFeb 23 17#51#17 The prepared software is set to be activated with process restartFeb 23 17#51#18 Start preparing software for local installationFeb 23 17#51#19 Action 1# install prepare action finished successfullyFeb 23 17#51#20 Action 2# install activate action startedFeb 23 17#51#20 The software will be activated with process restartFeb 23 17#51#22 Activating XR packagesFeb 23 17#51#58 0 processes affected at node 0/RP0/CPU0Feb 23 17#52#10 Action 2# install activate action finished successfullyFeb 23 17#52#11 Install operation 21 finished successfullyFeb 23 17#52#11 Ending operation 21Install add operation successfulXR Package StructureWhen you enter the Shell of the control plane (accessed by typing “bash” or “run” from the CLI) you are in a full Bash shell environment of the XR container with all the traditional Linux tools at your fingertips. (You can return to the IOS XR CLI by typing “exit”).Since all IOS XR packages used the RPM format, we will use the Linux rpm utility to inspect their content. The “-q” switch used below is used to query the installed packages database and the “-i” switch display the general information contained in the metadata of the RPM package.RP/0/RP0/CPU0#pwa-rtr#runWed Dec 2 03#04#32.231 UTC [xr-vm_node0_RP0_CPU0#~]$rpm -qi xrv9k-k9secName # xrv9k-k9sec Relocations# /opt/cisco/XR/packages/xrv9k-k9sec-1.0.0.0-r600Version # 1.0.0.0 Vendor# (none)Release # r600 Build Date# Thu Dec 24 08#46#14 2015Install Date# Fri Mar 4 12#18#15 2016 Build Host# iox-lnx-009Group # IOS-XR Source RPM# xrv9k-k9sec-1.0.0.0-r600.src.rpmSize # 8616918 License# Copyright (c) 2015 Cisco Systems Inc. All rights reserved.Signature # (none)Packager # alnguyenSummary # Bundle package for iosxr-securityArchitecture# x86_64Description #Bundle package for iosxr-securityBuild workspace# /auto/srcarchive16/production/6.0.0/xrv9k/workspaceWe can also use RPM utilities to query the requirement of the package (“-R” switch), this information is also in the RPM metadata and is crucial for dependency checking. In the example below, the k9sec package depends on three packages, each of them within a certain version range.[xr-vm_node0_RP0_CPU0#~]$rpm -qR xrv9k-k9sec/bin/sh/bin/sh/bin/sh/bin/shxrv9k-iosxr-fwding >= 1.0.0.0xrv9k-iosxr-fwding < 2.0.0.0xrv9k-iosxr-infra >= 1.0.0.0xrv9k-iosxr-infra < 2.0.0.0xrv9k-iosxr-os >= 1.0.0.0xrv9k-iosxr-os < 2.0.0.0All the Cisco packages are in a separate group named IOS-XR. With the “—queryformat” switch we specify the type of field we want to display and how we want to display them. The following command will query all the installed packages and display their groups and names we use the utility grep to filter the output and only display the package belonging to the IOS XR group. Enter the following command to query all the packages from that group#[xr-vm_node0_RP0_CPU0#~]$rpm -qa --queryformat '%{group} %{name}\\n' | grep IOS-XRIOS-XR xrv9k-spirit-bootIOS-XR xrv9k-iosxr-routingIOS-XR xrv9k-iosxr-fwdingIOS-XR xrv9k-iosxr-infraIOS-XR xrv9k-baseIOS-XR xrv9k-bgpIOS-XR xrv9k-common-pd-fib IOS-XR xrv9k-fwdingIOS-XR xrv9k-gcp-fwdingIOS-XR xrv9k-gdplaneIOS-XR xrv9k-iosxr-osIOS-XR xrv9k-os-supportIOS-XR xrv9k-k9secIOS-XR xrv9k-mgblWith the Help of standard Linux tools we can review the full history of installed IOS-XR packages#[xr-vm_node0_RP0_CPU0#~]$rpm -qa --queryformat '%{installtime} %{group} %{name}-%{version}-%{release} %{installtime#date}\\n' | grep IOS-XR | sort -nr | sed -e 's/^[0-9]*\\ IOS-XR\\ //'xrv9k-mgbl-2.0.0.0-r600 Fri Mar 4 12#33#59 2016xrv9k-k9sec-1.0.0.0-r600 Fri Mar 4 12#18#15 2016xrv9k-os-support-1.0.0.0-r600 Fri Jan 29 02#04#37 2016xrv9k-iosxr-os-1.0.0.0-r600 Fri Jan 29 02#04#35 2016xrv9k-gdplane-1.0.0.0-r600 Fri Jan 29 02#04#33 2016xrv9k-gcp-fwding-1.0.0.0-r600 Fri Jan 29 02#04#33 2016xrv9k-fwding-1.0.0.0-r600 Fri Jan 29 02#04#32 2016xrv9k-common-pd-fib-1.0.0.0-r600 Fri Jan 29 02#04#32 2016xrv9k-bgp-1.0.0.0-r600 Fri Jan 29 02#04#31 2016xrv9k-base-1.0.0.0-r600 Fri Jan 29 02#04#31 2016xrv9k-iosxr-infra-1.0.0.0-r600 Fri Jan 29 02#04#27 2016xrv9k-iosxr-fwding-2.0.0.0-r600 Fri Jan 29 02#04#25 2016xrv9k-spirit-boot-1.0.0.0-r600 Fri Jan 29 02#04#24 2016xrv9k-iosxr-routing-1.0.0.0-r600 Fri Jan 29 02#04#24 2016With the “—requires” switch we can query the RPM database and display the version of installed packages that are fulfilling the dependency of another package. Using the following command we learn which packages requires xrv9k-iosxr-routing to be present in the system#[xr-vm_node0_RP0_CPU0#~]$rpm -q --whatrequires xrv9k-iosxr-routingxrv9k-iosxr-fwding-2.0.0.0-r600.x86_64xrv9k-bgp-1.0.0.0-r600.x86_64xrv9k-mgbl-2.0.0.0-r600.x86_64With following command we get the version of the package that provides the xrv9k-iosxr-routing functionality#[xr-vm_node0_RP0_CPU0#~]$rpm -q --whatprovides xrv9k-iosxr-routing xrv9k-iosxr-routing-1.0.0.0-r600.x86_64Packages Installation from the ShellYou can use the install commands inside the Linux shell of the IOS-XR container to install packages using a shell script.[xr-vm_node0_RP0_CPU0#~]$#install update source http#//192.168.122.1#8080/xrv9k xrv9k-eigrp++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Update in progress...Scheme # httpHostname # 192.168.122.1#8080Collecting software state..Update packages #\txrv9k-eigrpFetching .... xrv9k-eigrp-1.0.0.0-r600.x86_64.rpm-6.0.0Adding packages \txrv9k-eigrp-1.0.0.0-r600.x86_64.rpm-6.0.0Mar 10 22#40#38 Install operation 24 started by root# install add source /misc/disk1/install_tmp_staging_area/6.0.0 xrv9k-eigrp-1.0.0.0-r600.x86_64.rpm-6.0.0 Mar 10 22#40#40 Install operation will continue in the backgroundMar 10 22#40#43 Install operation 24 finished successfullyInstall add operation successfulActivating xrv9k-eigrp-1.0.0.0-r600Mar 10 22#40#45 Install operation 30 started by root# install activate pkg xrv9k-eigrp-1.0.0.0-r600 Mar 10 22#40#45 Package list#Mar 10 22#40#45 xrv9k-eigrp-1.0.0.0-r600Mar 10 22#40#48 Install operation will continue in the backgroundMar 10 22#40#43 Install operation 24 finished successfullyInstall add operation successfulActivating xrv9k-eigrp-1.0.0.0-r600Mar 10 22#40#45 Install operation 25 started by root# install activate pkg xrv9k-eigrp-1.0.0.0-r600 Mar 10 22#40#45 Package list#Mar 10 22#40#45 xrv9k-eigrp-1.0.0.0-r600Mar 10 22#40#48 Install operation will continue in the backgroundMar 10 22#41#49 Install operation 25 finished successfullyAnalyzing Package Using LinuxAny Linux distribution installed with the RPM utilities allows you to look at the content of packages, this is very useful to analyze dependencies and verify package integrity outside of the router. In this example we use the “-l” switch to display the full path of all the files inside the package.NOTE# Notice the extra -p switch used to query uninstalled packages.cisco@compute#~$ cd ~/web_server/xrv9k/cisco@compute#~/web_server/xrv9k$ rpm -qpl xrv9k-k9sec-1.0.0.0-r600.x86_64.rpm-6.0.0//opt/opt/cisco/opt/cisco/XR/opt/cisco/XR/packages/opt/cisco/XR/packages/xrv9k-k9sec-1.0.0.0-r600/opt/cisco/XR/packages/xrv9k-k9sec-1.0.0.0-r600/all/opt/cisco/XR/packages/xrv9k-k9sec-1.0.0.0-r600/all/etc/opt/cisco/XR/packages/xrv9k-k9sec-1.0.0.0-r600/all/etc/compat-mdata<SNIP>cisco@compute#~/web_server/xrv9k$ rpm -qpR xrv9k-k9sec-1.0.0.0-r600.x86_64.rpm-6.0.0/bin/sh/bin/sh/bin/sh/bin/shxrv9k-iosxr-fwding >= 2.0.0.0xrv9k-iosxr-fwding < 3.0.0.0xrv9k-iosxr-infra >= 1.0.0.0xrv9k-iosxr-infra < 2.0.0.0xrv9k-iosxr-os >= 1.0.0.0xrv9k-iosxr-os < 2.0.0.0NOTE# Run these RPM utilities off-box on any Linux system that has the RPM utility installed. Experiment with some of the RPM commands to create your own dependency management tool for XR packages.—", "url": "/tutorials/2016-08-06-introduction-to-rpm/", "author": "Patrick Warichet", "tags": "iosxr, linux, rpm, yum" } , "tutorials-2016-08-26-working-with-ztp": { "title": "Working with Zero Touch Provisioning", "content": " IOS-XR# Working with ZTP Purpose of ZTP How ZTP works ZTP requirement DHCP server configuration HTTP Server requirement ZTP utilities ztp_helper.sh ZTP CLI Commands ZTP Logging Simple ZTP Example More Complex Example Purpose of ZTPZTP was designed to perform 2 different operations# Download and apply an initial configuration. Download and execute a shell script.How ZTP worksThe ZTP process is executed or invoked inside the control plane LXC Linux shell. Prior to IOS-XR 6.1.1 ZTP was executed within the default network namespace and could not access directly the data interfaces. Starting with IOS-XR 6.1.1, ZTP is executed inside the global-VRF network namespace with full access to all the data interfaces. This document is based on the IOS-XR 6.1.1 implementation.ZTP is launched from XR process manager (processmgr) when the system reaches level 999 (last processes to be scheduled for execution). At the beginning of its execution, ZTP will scan the configuration for the presence of a username. If there are no username configured, ZTP will fork a DHCP client on the management interface for IPv4 and IPv6 simultaneously, and wait for a response.If the DHCP response contains an option 67 (option 59 for IPv6), ZTP will download the file using the URI provided by option 67 (or option 59 for IPv6).If the file received is not a text file or the file is larger than 100 MB ZTP will erase the file and terminate its execution.Otherwise it will analyze the first line of the text file received, if the first line of the file starts with ”!! IOS XR”, it will consider it as a configuration file and pass it to the command line interpreter for syntax verification and commit it.If the first line starts with ”#!/bin/bash” or ”#!/bin/sh” ZTP will assume this is a script and start the execution, as illustrate below.If the text file cannot be interpreted as a shell script or a configuration file, ZTP will erase the file and terminate its execution.The script can use all the Linux tools available in the Control Plane LXC and perform addition HTTP GET using wget or curl for example to install package and/or download and apply configuration blocks. Note# In IOS-XR release 6.1.1 ZTP can also be invoked from the command line interpreter in this case it will start its execution even if a username or a configuration is present in the system.ZTP requirementZTP requires 2 external services# a DHCP server and an HTTP server, as illustrated above, the support for tftp has been dropped for security and reliability reasons.DHCP server configurationThe basic configuration described below provides a fixed IP address and a configuration file taking in account only the mac address of the management interface.host ncs-5001-rp0 { hardware ethernet e4#c7#22#be#10#ba; fixed-address 172.30.12.54; filename ~http#//172.30.0.22/configs/ncs5001-1.config~;}A more elaborate example that takes into account option 77 or option 15 for IPv6 (user-class) embedded in the dhcp request sent by the client, ZTP embed the string “exr-config” in the DHCP request as described below. The if statement also take into account the capability to re-image the system using iPXE (see iPXE deep dive document)host ncs-5001-rp0 { hardware ethernet e4#c7#22#be#10#ba; fixed-address 172.30.12.54; if exists user-class and option user-class = ~iPXE~ { filename = ~http#//172.30.0.22/boot.ipxe~; } elsif exists user-class and option user-class = ~exr-config~ { filename = ~http#//172.30.0.22/scripts/ncs-5001-rp0_ztp.sh~; }}Since ZTP does not require any intervention on the system, an easier way to provision the system is to use the serial number printed on the box and/or the RP, the configuration is then as follow#host ncs-5001-rp0 { option dhcp-client-identifier ~FOC1947R144~; fixed-address 172.30.12.54; if exists user-class and option user-class = ~iPXE~ { filename = ~http#//172.30.0.22/boot.ipxe~; } elsif exists user-class and option user-class = ~exr-config~ { filename = ~http#//172.30.0.22/scripts/ncs-5001-rp0_ztp.sh~; }}Note# The DHCP configuration examples described in this document use the open-source isc-dhcp-server syntax.HTTP Server requirementThe HTTP server should be reachable from the management interface or from a data interface if invoked manually and should be readable.ZTP utilitiesZTP includes a set of CLI commands and a set of shell utilities that can be sourced within the user script.ztp_helper.shztp_helper.sh is a shell script that can be sourced by the user script, it provides simple utilities to access some XR functionalities. These utilities are#xrcmd# Runs an XR exec commandxrcmd “show running”xrapply# Applies the block of configuration, specified in a file#cat >/tmp/config <<EOF!! XR config examplehostname plutoEOFxrapply /tmp/configxrapply_with_reason# Same as above, but specify a reason for commit history trackingcat >/tmp/config <<EOF!! XR config examplehostname saturnEOFxrapply_with_reason ~this is an important name change~ /tmp/configxrapply_string# Applies a block of configuration specified in a string. Use “\\n” to delimit line of configuration statement.xrapply_string ~hostname mars\\n interface TenGigE0/0/0/0\\n ipv4 address 172.30.0.144/24\\n”xrapply_string_with_reason# As above, but specify a reason for commit history tracking#xrapply_string_with_reason ”system renamed again~ ~hostname venus\\n interface TenGigE0/0/0/0\\n ipv4 address 172.30.0.144/24\\n”ZTP CLI Commandsztp initiate# Invokes a new ZTP DHCP session, logs will go to the console and /disk0#/ztp/ztp.logztp initiate allows the execution of a script even of the system has already been configured. This command is useful for testing ZTP without forcing a reload. This command is particularly useful to test scripts or if some manual operations are required before provisioning the box. With ZTP initiate you can specify any data interfaces on the system to be used for the whole ZTP process.RP/0/RP0/CPU0#venus#ztp initiate debug verbose interface TenGigE 0/0/0/0Invoke ZTP? (this may change your configuration) [confirm] [y/n] #ztp terminate# Terminates any ZTP session in progressRP/0/RP0/CPU0#venus#ztp terminate verboseMon Oct 10 16#52#38.507 UTCTerminate ZTP? (this may leave your system in a partially configured state) [confirm] [y/n] #yZTP terminatedztp breakout# Performs a 4x10 breakout detection on all 40 Gig interfaces, by default if no link is detected on any of the four 10Gig interfaces, the port will remain in 40 Gig mode.The subcommand nosignal-stay-in-breakout-mode will force the port in breakout mode even if there is no link signal detected but will place the interfaces in shutdown mode. The subcommand nosignal-stay-in-state-noshut will leave the port in breakout mode but will place the four 10Gig interface in no shutdown mode.The command “ztp breakout” may not be supported on the ASR9K routers.RP/0/RP0/CPU0#venus#ztp breakout ?apply XR configuration commands to apply(cisco-support)debug Run with additional logging to the console(cisco-support)hostname XR hostname to set(cisco-support)nosignal-stay-in-breakout-mode On no signal, prefer interfaces to remain in breakout mode(cisco-support)nosignal-stay-in-state-noshut On no signal, prefer interfaces to be noshut(cisco-support)verbose Run with logging to the console(cisco-support) ztp clean# Remove all ZTP files saved on diskRP/0/RP0/CPU0#venus#ztp clean verboseMon Oct 10 17#03#43.581 UTCRemove all ZTP temporary files and logs? [confirm] [y/n] #yAll ZTP files have been removed.If you now wish ZTP to run again from boot, do 'conf t/commit replace' followed by reload.ZTP LoggingZTP logs its operation on the flash file system in the directory /disk0#/ztp/. ZTP logs all the transaction with the DHCP server and all the state transition. Prior executions of ZTP are also logged in /disk0#/ztp/old_logs/Simple ZTP ExampleIn the following example we will review the execution of a simple configuration script downloaded from a data interface using the command “ztp initiate interface Ten 0/0/0/0 verbose”, this script will unshut all the interfaces of the system and configure a load interval of 30 sec on all of them.#!/bin/bash############################################################################## *** Be careful this is powerful and can potentially destroy your system ***# *** !!! Use at your own risk !!! ***## Script file should be saved on the backend HTTP server## Tested on NCS-5501 with IOS-XR 6.1.1##############################################################################source ztp_helper.shconfig_file=~/tmp/config.txt~interfaces=$(xrcmd ~show interfaces brief~)function activate_all_if(){ arInt=($(echo $interfaces | grep -oE '(Te|Fo|Hu)[0-9]*/[0-9]*/[0-9]*/[0-9]*')) for int in ${arInt[*]}; do echo -ne ~interface $int\\n no shutdown\\n load-interval 30\\n~ >> $config_file done xrapply_with_reason ~Initial ZTP configuration~ $config_file}### Script entry pointif [ -f $config_file ]; then /bin/rm -f $config_fileelse /bin/touch $config_filefiactivate_all_if;exit 0More Complex ExampleIn this example, the HTTP server hosts a CSV file that contains devices serial number followed by the hostname. The HTTP server also contains a basic configuration file that need to be applied. Finally a local repository accessible by HTTP contains IOS-XR and third party packages to be installed.After bootup ZTP will provide its serial number and query the back-end database using HTTP POST, once it obtains its hostname it will perform a version check, if the version on the system does not match the desired version, the system will change the boot order forcing a reboot using iPXE that will hopefully re-image the system to the desired version.If the system is running the correct version the script proceed by installing the k9sec package and create a generic RSA key for SSH. It will then add a local repository for third party packages and install midnight commander and its dependent packages. The script finishes its execution after downloading and applying the configuration.ZTP Script ncs-5001-rp0_ztp.sh#!/bin/bash############################################################################## *** Be careful this is powerful and can potentially destroy your system ***# *** !!! Use at your own risk !!! ***## Script file should be saved on the backend HTTP server## Tested on NCS-5501 with IOS-XR 6.1.1##############################################################################export LOGFILE=/disk0#/ztp/user-script.logexport HTTP_SERVER=http#//172.30.0.22export SYSLOG_SERVER=172.30.0.22export SYSLOG_PORT=514export CONFIG_PATH=configsexport SCRIPT_PATH=scriptsexport RPM_PATH=~packages/ncs5k/6.1.1~export PHP_SCRIPT=~php/device_name.php~export DESIRED_VER=~6.1.1~K9SEC_RPM=ncs5k-k9sec-3.1.0.0-r611.x86_64.rpm## ztp_helper is inside the Fretta Code-base - ASSUMPTIONsource ztp_helper.shfunction ztp_log() { # Sends logging information to local file and syslog server syslog ~$1~ echo ~$(date +~%b %d %H#%M#%S~) ~$1 >> $LOGFILE}function syslog() { # Sends syslog messages with netcat (nc) echo ~ztp-script# ~$1 | nc -u -q1 $SYSLOG_SERVER $SYSLOG_PORT}function get_hostname(){ # Uses serial number to query a remote database and get hostname using HTTP POST local sn=$(dmidecode | grep -m 1 ~Serial Number#~ | awk '{print $NF}'); local result=~`wget -O- --post-data=~serial=$sn~ ${HTTP_SERVER}/${PHP_SCRIPT}`~; if [ ~$result~ != ~Not found~ ]; then DEVNAME=$result; return 0 else \tztp_log ~Serial $sn not found, hostname not set~; return 1 fi}function download_config(){\t# Downloads config using hostname ztp_log ~### Downloading system configuration ###~; /usr/bin/wget ${HTTP_SERVER}/${CONFIG_PATH}/${DEVNAME}.config -O /disk0#/new-config 2>&1 >> $LOGFILE if [[ ~$?~ != 0 ]]; then \tztp_log ~### Error downloading system configuration ###~ else ztp_log ~### Downloading system configuration complete ###~; fi}function apply_config(){\t# Applies initial configuration ztp_log ~### Applying initial system configuration ###~; xrapply_with_reason ~Initial ZTP configuration~ /disk0#/new-config 2>&1 >> $LOGFILE; ztp_log ~### Checking for errors ###~; local config_status=$(xrcmd ~show configuration failed~); if [[ $config_status ]]; then \techo $config_status >> $LOGFILE ztp_log ~!!! Error encounter applying configuration file, review the log !!!!~; fi ztp_log ~### Applying system configuration complete ###~;}function install_k9sec_pkg(){ # Installs the k9sec package from repository, create a RSA key modulus 1024 ztp_log ~### XR K9SEC INSTALL ###~ /usr/bin/wget ${HTTP_SERVER}/${RPM_PATH}/${K9SEC_RPM} -O /disk0#/$K9SEC_RPM 2>&1 if [[ ~$?~ != 0 ]]; then ztp_log ~### Error downloading $K9SEC_RPM ###~ else ztp_log ~### Downloading $K9SEC_PKG complete ###~; fi xrcmd ~install update source /disk0#/ $K9SEC_RPM~ 2>&1 >> $LOGFILE local complete=0 while [ ~$complete~ = 0 ]; do complete=`xrcmd ~show install active~ | grep k9sec | head -n1 | wc -l` ztp_log ~Waiting for k9sec package to be activated~ sleep 5 done if [[ -z $(xrcmd ~show crypto key mypubkey rsa~) ]]; then echo ~1024~ | xrcmd ~crypto key generate rsa~ else echo -ne ~yes\\n 1024\\n~ | xrcmd ~crypto key generate rsa~ fi rm -f /disk0#/$K9SEC_RPM ztp_log ~### XR K9SEC INSTALL COMPLETE ###~}function check_version(){\t# returns 0 is version matches, 1 otherwise local current_ver=`xrcmd ~show version~ | grep Version | grep Cisco | cut -d ~ ~ -f 6`; ztp_log ~current=$current_ver, desired=$DESIRED_VER~; if [ ~$DESIRED_VER~ = ~$current_version~ ]; then return 0 else return 1 fi}function reboot_ipxe(){ # Do not use in production, may not be supported on NCS-5508 ztp_log ~### Mounting EFIvar and changing boot order~ local EFI_FILESYS=~/sys/firmware/efi/efivars~ if [ ! -d $EFI_FILESYS ]; then ztp_log ~EFI mount point not present~ fi /bin/mount -t efivarfs efivarfs $EFI_FILESYS if [[ ~$?~ != 0 ]]; then ztp_log ~Error mounting efivars filesystem~; fi local iPXE=$(/usr/sbin/efibootmgr | grep IPXE | awk -v FS=~(Boot|*)~ '{print $2}') local MMC=$(/usr/sbin/efibootmgr | grep ~HS-SD/MMC~ | awk -v FS=~(Boot|*)~ '{print $2}') local Shell=$(/usr/sbin/efibootmgr | grep Shell | awk -v FS=~(Boot|*)~ '{print $2}') /usr/sbin/efibootmgr -o $iPXE,$MMC,$Shell if [[ ~$?~ != 0 ]]; then ztp_log ~Error changing boot order~; fi ztp_log ~### Resetting the system~; echo 1 > /proc/sys/kernel/sysrq echo b > /proc/sysrq-trigger ztp_log ~Unable to reset the system~;}function add_repo(){\t# add a local repo to yum package manager /usr/bin/yum-config-manager --add-repo ${HTTP_SERVER}/${RPM_PATH} 2>&1}function install_mc(){\t# uses yum to install packages and dependant package(s) automatically if /usr/bin/yum list installed ~mc~ >/dev/null 2>&1; then \tztp_log ~### Installing midnight commander~; /usr/bin/yum install mc -y 2>&1 else \tztp_log ~### Midnight commander already installed ###~ fi}# ==== Script entry point ====get_hostname;if [[ ~$?~ != 0 ]]; then ztp_log ~No valid hostname found terminating ZTP~; exit 1fiztp_log ~Hello from ${DEVNAME} !!!~;check_version;if [[ ~$?~ != 0 ]]; then ztp_log ~Version mismatch, we will upgrade using iPXE~; reboot_ipxe;else ztp_log ~Version match, proceeding to configuration~;fiztp_log ~Starting autoprovision process...~;install_k9sec_pkg;add_repo;install_mc;download_config;apply_config;ztp_log ~Autoprovision complete...~;exit 0Backend PHP script device_name.php<?php$file = 'inventory.txt';$searchfor = ($_POST['serial']);// the following line prevents the browser from parsing this as HTML.header('Content-Type# text/plain');// get the file contents, assuming the file to be readable (and exist)$contents = file_get_contents($file);// escape special characters in the query$pattern = preg_quote($searchfor, '/');// finalise the regular expression, matching the whole line$pattern = ~/^.*$pattern.*\\$/m~;// search, and store first matching occurences in $matchesif(preg_match($pattern, $contents, $matches)){ //match the last element of the line preg_match('/[^,]*$/', $matches[0], $results); echo $results[0];}else{ echo ~Not found~;}?>CSV file inventory.txtFOC2647D246,ncs-5001-aFOC1568P682,ncs-5001-bFOC1947R143,ncs-5001-cLogging outputOct 11 11#05#38 172.30.0.54 ztp-script# Hello from ncs-5001-c !!!Oct 11 11#05#40 172.30.0.54 ztp-script# current=6.1.1, desired=6.1.1Oct 11 11#05#40 172.30.0.54 ztp-script# Version match, proceeding to configurationOct 11 11#05#41 172.30.0.54 ztp-script# Starting autoprovision process...Oct 11 11#05#42 172.30.0.54 ztp-script# ### XR K9SEC INSTALL ###Oct 11 11#05#44 172.30.0.54 ztp-script# ### Downloading complete ###Oct 11 11#05#55 172.30.0.54 ztp-script# Waiting for k9sec package to be activatedOct 11 11#06#01 172.30.0.54 ztp-script# ### XR K9SEC INSTALL COMPLETE ###Oct 11 11#06#03 172.30.0.54 ztp-script# ### Installing midnight commander ###Oct 11 11#06#04 172.30.0.54 ztp-script# ### Downloading system configuration ###Oct 11 11#06#05 172.30.0.54 ztp-script# ### Downloading system configuration complete ###Oct 11 11#06#06 172.30.0.54 ztp-script# ### Applying initial system configuration ###Oct 11 11#06#11 172.30.0.54 ztp-script# !!! Checking for errors !!!Oct 11 11#06#14 172.30.0.54 ztp-script# ### Applying system configuration complete ###Oct 11 11#06#15 172.30.0.54 ztp-script# Autoprovision complete...", "url": "/tutorials/2016-08-26-working-with-ztp/", "author": "Patrick Warichet", "tags": "iosxr, cisco" } , "#": {} , "#": {} }