PfR:Solutions:EnterpriseWAN

From DocWiki

Revision as of 10:30, 26 November 2013 by Jbarozet (Talk | contribs)
Jump to: navigation, search

Pfr-banner-solutions.png



PfR Enterprise WAN Application Control


Navigation


Contents




Version Used - PfR Version 2

IOS 15.2(3)T and IOS-XE 3.6 releases are key milestones for PfR. Defaults have changed to better align with the customer deployments and the PfR Best practices.
Among the most important changes are:

  • Mode route control is on by default
  • Resolvers range and utilization have been removed. Load-balancing is used as the last resolver and cannot be disabled.
  • Automatic learning is enabled by default
  • Learning monitor-period is 1, and periodic-interval is 0 by default

See PfR Simplification for more information.

This guide is based on IOS 15.3(3)M.


Intelligent WAN (IWAN)

The Cisco Intelligent WAN (IWAN) is a system that enhances collaboration and cloud application performance while reducing the operating cost of the WAN. IWAN leverages low-cost high-bandwidth Internet services to increase bandwidth capacity without compromising performance, availability or security of collaboration or cloud based applications. Organizations can use IWAN to leverage the Internet as a WAN transport, as well as, for direct access to Public Cloud applications. (See Figure 1.)

Figure 1. Cisco IWAN works with both private and public clouds.

Pfr-iwan-fig1.png


Cisco Intelligent WAN is based on four design components:

  • Transport-Independent Design
  • Intelligent Path Control
  • Application Optimization
  • Secure Internet Access


Transport-Independent Design simplifies the WAN design by using an IPsec VPN overlay over all WAN transport options including MPLS, Internet, and Cellular (3G/4G). The single VPN overlay reduces routing and security complexity, and provides flexibility in choosing providers and transport options. Cisco Dynamic Multipoint VPN (DMVPN) provides the IWAN IPsec overlay. Two or more WAN transport providers are recommended to increase network availability up to 99.999%.


Intelligent Path Control with Cisco Performance Routing (PfR) improves application delivery and WAN efficiency. PfR protects business applications from fluctuating WAN performance while intelligently load balancing traffic over all WAN paths. PfR monitors the network performance (delay, jitter, packet loss,…) to forward critical applications over the best performing paths based on the application policy. PfR’s advanced load balancing evenly distributes traffic to maintain equivalent link utilization levels ‒ even over links with differing bandwidth capacities. IWAN Intelligent Path Control is key to providing a business-class WAN over Internet transports. (See Figure 2.)

Figure 2. Cisco IWAN Intelligent Path Control.

Pfr-iwan-fig2.png


Application Optimization is provided by Cisco Application Visibility and Control (AVC) and Cisco Wide Area Application Services (WAAS). With applications becoming increasingly “opaque” (due to increased use of HTTP-based applications), static port classification of applications is no longer sufficient. AVC makes IWAN application-aware with deep packet inspection of traffic to identify and monitor application performance. With increased visibility into the applications on the network, better Quality of Service (QoS) policies can be enabled and fine-tuned to ensure that critical applications are properly prioritized across the network. Cisco WAAS provides application-specific acceleration capabilities that improve response times while reducing WAN bandwidth requirements.


Secure Internet Access offloads user traffic destined for Public Cloud or Internet out the local Internet service. This improves Public Cloud application performance while reducing traffic over the WAN. Cisco’s Cloud Web Security (CWS) service provides a cloud based web proxy to centrally manage and secure user traffic accessing the Internet.


For More Information Read about IWAN at www.cisco.com/go/iwan or contact your local Cisco account representative.



Enterprise Needs

Presentation

This solution is for an Enterprise which is dual-attached to two Service Providers. Multiple scenarios are covered like:

  • Dual MPLS-VPN service
  • or a primary MPLS-VPN and a secondary DMVPN over the Public Internet.
  • or dual DMVPN over the Public Internet

Pfr-iwan-deployments.png


PfR being a key pillar of the IWAN initiative, this guide will focus on a design with dual DMVPN clouds.


In this guide we will cover several concepts including:

  • Building the tunnels with DMVPN
  • Target Discovery to simplify the active mode configuration
  • Advanced learning: defining groups of traffic classes, e.g. VOICE_VIDEO, CRITICAL and BEST_EFFORT using a PfR feature called 'learn-list'
  • Measurement modes: passive for BEST_EFFORT and active for VOICE_VIDEO and CRITICAL
  • Defining specific policies for each group.
  • Path Preference, also called Link-groups


Traffic Classification and Overall Policies

In this guide, the enterprise traffic is divided into 3 main application groups (Service Class) that have their own policies. This is just an example, one can adjust based on his requirements but that's a good starting point.


Voice and Video traffic - VOICE_VIDEO Service Class

  • Voice and video traffic is running between the central site and the branches. This traffic is already marked with DSCP EF and the transport is RTP. To capture video we also match DSCP AF41 and CS4 which is used by Tandberg endpoints.
  • We would like to track delay, jitter and loss.
  • We want to have deterministic path selection and therefore use the primary SP called SP1.
  • If the primary WAN experiences brownouts or blackout, we want PfR to failover this traffic to the secondary path.


Critical Application - CRITICAL Service Class

  • There is a highly important application running between branches and the central site network and running over TCP.
  • This traffic is marked with DSCP AF31.
  • This application is interactive, so it is sensitive to delay. In addition, it does not respond well to packet loss. The servers being used for this important application are also responsible for other applications that are not only less important, but that have different requirements of the network.
  • We want to have deterministic path selection and therefore use the primary SP SP1.
  • If the primary WAN experiences brownouts or blackout, we want PfR to failover this traffic to the secondary path.


Best Effort Applications - BEST_EFFORT Service Class

  • The rest of the traffic is just HTTP or email and should be load balanced over all exit interfaces, SP1 and SP2.


Qos is already in place with classification/marking procedures. Therefore packets entering on the Border Router are already classified and marked directly on the access switch connecting the station, IP Phone or multimedia terminal. That means PfR can use the DSCP field as an efficient way for the traffic profiling phase. This guide will use the DSCP values as the classification criteria. But you could also use a combination of destination prefixes and DSCP or even NBAR (see [[PfR:Solutions:NBAR | PfR NBAR Based Application Control] for more information). If the traffic is not marked when entering the BR, then a good option is to classify on ingress, mark the DSCP and then use the DSCP as the classification criteria within PfR. Classification can be done with the usual use of access-lists.



PfR Network Topology Used

Transport and Overlay Backbones

Addressing Plan:

  • 172.16.0.0/16 - Transport
  • 10.0.0.0/16 - Overlay - DMVPN1 (10.0.100.0/24) and DMVPN2 (10.0.200.0/24)
  • 10.1.0.0/16 - Branch inside prefixes
  • 10.2.0.0/16 - Branch loopbacks
  • 10.10.0.0/16 - Central Sites


Central site (10.10.0.0/16):

  • A dedicated Master Controller (MC) R3 and two Border Routers (BRs) R4 and R5
  • R2 is a campus core router or L3 switch and used as an IP SLA responder shadow router.
  • Server S1: HTTP and Mail server, voice peer.


Branch site R9 (10.1.9.0/24):

  • R9 with a combo of Master Controller and Border Router
  • S12: HTTP and Mail client, voice peer


Branch site R10 (10.1.10.0/24):

  • R10 with a combo of Master Controller and Border Router
  • S11: HTTP and Mail client, voice peer


Two SP Clouds (SP1 and SP2) with a delay generator (D7).


The Transport Topology is based on two Service Providers, SP1 being considered as the primary one with known SLAs. Therefore SP1 is the preferred path for voice/video and critical applications:


Pfr-lab3-topology1.jpg


The overlay topology is based on two DMVPN clouds:


Pfr-lab3-topology2.jpg


This overlay design with two DMVPN clouds can accommodate any kind of transports. The primary path can connect to an MPLS-VPN or even to the Public Internet. The configuration of PfR (and QoS) will remain the same even if the transport design changes.


Routers and Servers used

Servers:

  • S1 is a server on the central site.
  • S11 and S12 are clients on their branch office


Routers:

  • R3 is the MC for the central site
  • R4 and R5 are the BRs for the central site
  • R9 and R10 are MC/BR for their branch offices
  • R2 is just a router or L3 switch in the central site. It's the campus core backbone.


Delay Generator

  • R7 is a delay generator


Traffic Generation

Traffic generation between R1 and R11/R12:

  • Voice traffic on RTP – DSCP EF between spokes and the hub but also between spokes
  • Business applications running on port TCP 7000 – DSCP AF31 between spokes and hub
  • Best effort application – DSCP 0 between spokes and hub


Traffic details:


BEST EFFORT

http

  • dest-ip: 10.10.1.1
  • src-ip: 10.1.9.1 to 10.1.9.100 by 0.0.0.1 (branch9)
  • src-ip: 10.1.10.1 to 10.1.10.100 by 0.0.0.1 (branch10)
  • TCP port 80
  • DSCP 0

smtp

  • dest-ip: 10.10.2.1
  • src-ip: 10.1.9.2 to 10.1.9.100 by 0.0.0.1 (branch9)
  • src-ip: 10.1.10.2 to 10.1.10.100 by 0.0.0.1 (branch10)
  • TCP port 25


BUSINESS

http

  • dest-ip: 10.10.3.1
  • src-ip: 10.1.9.3 to 10.1.9.100 by 0.0.0.1 (branch9)
  • src-ip: 10.1.10.3 to 10.1.10.100 by 0.0.0.1 (branch10)
  • TCP port 7000
  • DSCP = AF31 (26, 0x1A)
  • TOS = 0x68, 104


VOICE

rtp

  • 10.1.10.200 (branch10) <----> 10.10.1.1 (hub)
  • 10.1.9.200 (branch9) <----> 10.10.1.1 (hub)
  • 10.1.9.200 (branch9) <----> 10.1.10.200 (branch10)
  • DSCP = EF (46, 0x2E)
  • TOS = 0xB8, 184
  • dest-port 20000
  • src-port 10000



Dual DMVPN Setup

DMVPN Phase Summary

DMVPN has multiple phases that are summarized below:

Pfr-dmvpn-phases.png


DMVPN Phase 2 has no summarization on the hub:

  • Each spoke has the next-hop (spoke address) for each spoke destination prefix.
  • PfR has all information to enforce the path with dynamic PBR and the correct next-hop information


DMVPN phase3 allows route summarization:

  • When parent route lookup is performed, only the route to the hub is available
  • NHRP dynamically installs shortcut tunnel and hence populates RIB/CEF.
  • PfR still has the hub next-hop information and is currently unaware of the next-hop change.


PfR currently supports DMVPN Phase 2 only.


DMVPN Configuration

PfR runs over Dual DMVPN clouds. On the hub site R4 is the hub for DMVPN1 and R5 is the hub for DMVPN2. Each spoke as 2 tunnels, one per DMVPN cloud.

R4 DMVPN configuration:

! 
! ------------------------------------------------------------------------
! 1. Configure the crypto keyring
! The crypto keyring defines a pre-shared key (or password) valid for IP sources 
! reachable within a particular VRF. 
! This key is a wildcard pre-shared key if it applies to any IP source. 
! A wildcard key is configured using the 0.0.0.0 0.0.0.0 network/mask combination.
!
crypto keyring DMVPN  
  pre-shared-key address 0.0.0.0 0.0.0.0 key pfr 
!
!
! ------------------------------------------------------------------------
! 2. Configure the ISAKMP policy
! The ISAKMP policy for DMVPN uses the following: 
!   - Advanced Encryption Standard (AES) 
!   - Authentication by pre-shared key
!
crypto isakmp policy 10
 encr aes
 authentication pre-share
!
crypto isakmp invalid-spi-recovery
!
!
! ------------------------------------------------------------------------
! 3. Define the IPSec transform set
! A transform set is an acceptable combination of security protocols, algorithms, 
! and other settings to apply to IPsec-protected traffic. 
! Peers agree to use a particular transform set when protecting a particular data flow.
!
crypto ipsec transform-set DMVPN esp-aes 256 esp-sha-hmac 
 mode transport
!
!
! ------------------------------------------------------------------------
! 4. Create the IPSec profile
! The IPsec profile creates an association between an ISAKMP profile and an IPsec transform-set.
!
crypto ipsec profile DMVPN
 set transform-set DMVPN 
!
!
! ------------------------------------------------------------------------
! 5. Configure the mGRE tunnel 
!   - Configure basic interface settings
!      The IP MTU should be configured to 1400 
!      The ip tcp adjust-mss should be configured to 1360. 
!      There is a 40 byte difference which corresponds to the combined IP and TCP header length.
!   - Configure NHRP
!   - Set NHRP holdtime to 600
!   - Set Delay to 1000
!   - Apply the IPSec profile to the tunnel
!
interface Tunnel100
 bandwidth 1000
 ip address 10.0.100.4 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication cisco
 ip nhrp map multicast dynamic
 ip nhrp network-id 1
 ip nhrp holdtime 600
 ip tcp adjust-mss 1360
 load-interval 30
 delay 1000
 tunnel source Ethernet0/1
 tunnel mode gre multipoint
 tunnel key 100
 tunnel protection ipsec profile DMVPN
!     


R5 DMVPN configuration:

! 
crypto keyring DMVPN  
  pre-shared-key address 0.0.0.0 0.0.0.0 key pfr 
!
crypto isakmp policy 10
 encr aes
 authentication pre-share
!
crypto isakmp invalid-spi-recovery
!
!
crypto ipsec transform-set DMVPN esp-aes 256 esp-sha-hmac 
 mode transport
!
crypto ipsec profile DMVPN
 set transform-set DMVPN 
!
interface Tunnel200
 bandwidth 1000
 ip address 10.0.200.5 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication cisco
 ip nhrp map multicast dynamic
 ip nhrp network-id 2
 ip nhrp holdtime 600
 ip tcp adjust-mss 1360
 load-interval 30
 delay 1000
 tunnel source Ethernet0/1
 tunnel mode gre multipoint
 tunnel key 200
 tunnel protection ipsec profile DMVPN
!


R9 Spoke Configuration:

! 
crypto keyring DMVPN  
  pre-shared-key address 0.0.0.0 0.0.0.0 key pfr 
!
crypto isakmp policy 10
 encr aes
 authentication pre-share
!
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 10
!
crypto ipsec transform-set DMVPN esp-aes 256 esp-sha-hmac 
 mode transport
!
crypto ipsec profile DMVPN
 set transform-set DMVPN 
!
!
! -----------------------------------------
! DMVPN1 Tunnel
!
interface Tunnel100
 ip address 10.0.100.9 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication cisco
 ip nhrp map 10.0.100.4 172.16.41.4
 ip nhrp map multicast 172.16.41.4
 ip nhrp network-id 1
 ip nhrp holdtime 600
 ip nhrp nhs 10.0.100.4
 ip nhrp registration timeout 60
 ip tcp adjust-mss 1360
 delay 1000
 tunnel source Ethernet0/1
 tunnel mode gre multipoint
 tunnel key 100
 tunnel protection ipsec profile DMVPN
!
! -----------------------------------------
! DMVPN2 Tunnel
!
interface Tunnel200
 ip address 10.0.200.9 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication cisco
 ip nhrp map 10.0.200.5 172.16.52.5
 ip nhrp map multicast 172.16.52.5
 ip nhrp network-id 2
 ip nhrp holdtime 600
 ip nhrp nhs 10.0.200.5
 ip nhrp registration timeout 60
 ip tcp adjust-mss 1360
 delay 1000
 tunnel source Ethernet0/2
 tunnel mode gre multipoint
 tunnel key 200
 tunnel protection ipsec profile DMVPN
!


R10 Spoke Configuration:

!
crypto keyring DMVPN
  pre-shared-key address 0.0.0.0 0.0.0.0 key pfr
!
crypto isakmp policy 10
 encr aes
 authentication pre-share
!
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 10
!
crypto ipsec transform-set DMVPN esp-aes 256 esp-sha-hmac
 mode transport
!
crypto ipsec profile DMVPN
 set transform-set DMVPN
!
!
! -----------------------------------------
! DMVPN1 Tunnel
!
interface Tunnel100
 ip address 10.0.100.10 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication cisco
 ip nhrp map 10.0.100.4 172.16.41.4
 ip nhrp map multicast 172.16.41.4
 ip nhrp network-id 1
 ip nhrp holdtime 600
 ip nhrp nhs 10.0.100.4
 ip nhrp registration timeout 60
 ip tcp adjust-mss 1360
 delay 1000
 tunnel source Ethernet0/1
 tunnel mode gre multipoint
 tunnel key 100
 tunnel protection ipsec profile DMVPN
!
! -----------------------------------------
! DMVPN2 Tunnel
!
interface Tunnel200
 ip address 10.0.200.10 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication cisco
 ip nhrp map 10.0.200.5 172.16.52.5
 ip nhrp map multicast 172.16.52.5
 ip nhrp network-id 2
 ip nhrp holdtime 600
 ip nhrp nhs 10.0.200.5
 ip nhrp registration timeout 60
 ip tcp adjust-mss 1360
 delay 1000
 tunnel source Ethernet0/2
 tunnel mode gre multipoint
 tunnel key 200
 tunnel protection ipsec profile DMVPN
!
!


Routing on the Overlay Backbone

R4 and R5 are iBGP peers and implement a feature called Dynamic Neighbors.
BGP dynamic neighbor support allows BGP peering to a group of remote neighbors that are defined by a range of IP addresses. Each range can be configured as a subnet IP address.

Pfr-bgp-dynamic.jpg


With this feature R4 and R5 just listen to incoming BGP connections. This avoids the manual configuration of all remote sites neighbors.


R4 Hub Configuration:

!
router bgp 10
 bgp router-id 10.10.10.4
 bgp cluster-id 10.10.10.4
 bgp log-neighbor-changes
 bgp listen range 10.0.100.0/24 peer-group SPOKES-1
 neighbor SPOKES-1 peer-group
 neighbor SPOKES-1 remote-as 10
 neighbor SPOKES-1 update-source Loopback0
 neighbor SPOKES-1 timers 5 25
 neighbor 10.10.10.5 remote-as 10
 neighbor 10.10.10.5 update-source Loopback0
 !
 address-family ipv4
  bgp redistribute-internal
  network 10.0.100.0 mask 255.255.255.0
  network 10.10.10.4 mask 255.255.255.255
  aggregate-address 10.10.0.0 255.255.0.0 summary-only
  neighbor SPOKES-1 activate
  neighbor SPOKES-1 route-reflector-client
  neighbor SPOKES-1 route-map SET-LP in
  neighbor SPOKES-1 route-map SET-LP out
  neighbor 10.10.10.5 activate
  maximum-paths ibgp 4
  distance bgp 20 21 21
 exit-address-family
!
route-map SET-LP permit 10
 set local-preference 200
!
!

Notes:

  • All spokes are iBGP peers
  • R4 listens from incoming connections from range 10.0.100.0/24
  • R4 summarizes hub prefix to 10.10.0.0/16
  • R4 does NOT summarize spoke prefixes 10.1.0.0/16
  • A route-map is used to set the Local Preference to 200 - this DMVPN is supposed to be the primary.
  • ibgp multipath is used.


R5 Hub Configuration:

!
router bgp 10
 bgp router-id 10.10.10.5
 bgp cluster-id 10.10.10.5
 bgp log-neighbor-changes
 bgp listen range 10.0.200.0/24 peer-group SPOKES-2
 neighbor SPOKES-2 peer-group
 neighbor SPOKES-2 remote-as 10
 neighbor SPOKES-2 update-source Loopback0
 neighbor SPOKES-2 timers 5 25
 neighbor 10.10.10.4 remote-as 10
 neighbor 10.10.10.4 update-source Loopback0
 !
 address-family ipv4
  bgp redistribute-internal
  network 10.0.200.0 mask 255.255.255.0
  network 10.10.10.5 mask 255.255.255.255
  aggregate-address 10.10.0.0 255.255.0.0 summary-only
  neighbor SPOKES-2 activate
  neighbor SPOKES-2 route-reflector-client
  neighbor SPOKES-2 route-map SET-LP in
  neighbor SPOKES-2 route-map SET-LP out
  neighbor 10.10.10.4 activate
  maximum-paths ibgp 4
  distance bgp 20 21 21
 exit-address-family
!
!
route-map SET-LP permit 10
 set local-preference 50
!

Notes:

  • All spokes are iBGP peers
  • R5 listens from incoming connections from range 10.0.200.0/24
  • R5 summarizes hub prefix to 10.10.0.0/16
  • R5 does NOT summarize spoke prefixes 10.1.0.0/8
  • A route-map is used to set the Local Preference to 50 - this DMVPN is supposed to be the secondary (backup)
  • ibgp multipath is used.


R9 Spoke Configuration:

!
router bgp 10
 bgp log-neighbor-changes
 neighbor HUBS-1 peer-group
 neighbor HUBS-1 remote-as 10
 neighbor HUBS-1 update-source Tunnel100
 neighbor HUBS-1 timers 5 25
 neighbor HUBS-2 peer-group
 neighbor HUBS-2 remote-as 10
 neighbor HUBS-2 update-source Tunnel200
 neighbor HUBS-2 timers 5 25
 neighbor 10.0.100.4 peer-group HUBS-1
 neighbor 10.0.200.5 peer-group HUBS-2
 !
 address-family ipv4
  network 10.0.100.0 mask 255.255.255.0
  network 10.0.200.0 mask 255.255.255.0
  network 10.1.9.0 mask 255.255.255.0
  network 10.2.9.9 mask 255.255.255.255
  neighbor 10.0.100.4 activate
  neighbor 10.0.200.5 activate
  maximum-paths ibgp 4
 exit-address-family
!

Notes:

  • All spokes are BGP Route Reflector clients
  • ibgp multipath is used.



Check Routing

Routing was defined so that DMVPN is the primary path. So, PfR being not active, the parent routes are BGP based on R4 and R5 and R4 is the preferred exit point (higher local preference applied on ingress on R4) as seen below:

On R4 (hub) - Route to destination prefixes BRANCH2 (10.1.10.0/24):


R4#sh ip bgp 10.1.10.0
BGP routing table entry for 10.1.10.0/24, version 34
Paths: (1 available, best #1, table default)
Multipath: iBGP
  Advertised to update-groups:
     45         47
  Refresh Epoch 1
  Local, (Received from a RR-client)
    10.0.100.10 from *10.0.100.10 (10.2.10.10)
      Origin IGP, metric 0, localpref 200, valid, internal, best
      rx pathid: 0, tx pathid: 0x0
R4#

What to check:

  • Route to branch has local preference 200 on DMVPN1
  • Best route is directly DMVPN1


On R5 (hub) - Route to destination prefixes BRANCH2 (10.1.10.0/24):


R5#sh ip bgp 10.1.10.0
BGP routing table entry for 10.1.10.0/24, version 20
Paths: (2 available, best #2, table default)
Multipath: iBGP
  Advertised to update-groups:
     46
  Refresh Epoch 1
  Local, (Received from a RR-client)
    10.0.200.10 from *10.0.200.10 (10.2.10.10)
      Origin IGP, metric 0, localpref 50, valid, internal
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  Local
    10.0.100.10 (metric 11) from 10.10.10.4 (10.10.10.4)
      Origin IGP, metric 0, localpref 200, valid, internal, best  <--- Best Route
      Originator: 10.2.10.10, Cluster list: 10.10.10.4
      rx pathid: 0, tx pathid: 0x0
R5#

What to check:

  • Route to branch has local preference 200 on DMVPN1 from R4
  • Route to branch has local preference 50 on DMVPN2.
  • Best route is through R4 (DMVPN1)


On R9 (spoke) - Routes to the hub:


R9#sh ip bgp 10.10.0.0
BGP routing table entry for 10.10.0.0/16, version 12
Paths: (2 available, best #1, table default)
Multipath: iBGP
  Not advertised to any peer
  Refresh Epoch 3
  Local, (aggregated by 10 10.10.10.4)
    10.0.100.4 from 10.0.100.4 (10.10.10.4)
      Origin IGP, metric 0, localpref 200, valid, internal, atomic-aggregate, best
      rx pathid: 0, tx pathid: 0x0
  Refresh Epoch 3
  Local, (aggregated by 10 10.10.10.5)
    10.0.200.5 from 10.0.200.5 (10.10.10.5)
      Origin IGP, metric 0, localpref 50, valid, internal, atomic-aggregate
      rx pathid: 0, tx pathid: 0
R9#

What to check:

  • Hub subnets 10.10.0.0/16 advertized through DMVPN1 (10.0.100.4) and DMVPN2 (10.0.200.5)
  • Route from R4 preferred because of highest local preference


On R9 (spoke) - Routes to the other spoke BRANCH2 (R10):


R9#sh ip bgp 10.1.10.0
BGP routing table entry for 10.1.10.0/24, version 4
Paths: (2 available, best #2, table default)
Multipath: iBGP
  Not advertised to any peer
  Refresh Epoch 3
  Local
    10.0.100.10 from 10.0.200.5 (10.10.10.5)
      Origin IGP, metric 0, localpref 200, valid, internal
      Originator: 10.2.10.10, Cluster list: 10.10.10.5, 10.10.10.4
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 3
  Local
    10.0.100.10 from 10.0.100.4 (10.10.10.4)
      Origin IGP, metric 0, localpref 200, valid, internal, best
      Originator: 10.2.10.10, Cluster list: 10.10.10.4
      rx pathid: 0, tx pathid: 0x0
R9#

What to check:

  • R10 route advertized through R4 (DMVPN1, 10.0.100.4) - best route because of higher local preference
  • R10 route advertized through R5 but through DMVPN1 (10.0.100.10) and therefore R4 (check cluster list) - Reason is again higher local preference given to DMVPN1.
  • Local Preference 200, traffic to 10.10.0.0/16 through DMVPN2 by default. PfR will change this behavior



Checking flows

In this section we are using Flexible NetFlow on R2 to look at the traffic and check what is the current path used.

Create the flow record:

!*******************************
! FLOW RECORD
! What data do I want to meter?
! First define the flow record that you want.
! ‘match’ is used to identify key fields, ie, those fields used to define what a flow is
! ‘collect’ is used to define what fields we want to monitor
!
flow record RECORD-STATS
 match ipv4 dscp
 match ipv4 protocol
 match ipv4 source address
 match ipv4 destination address
 match transport source-port
 match transport destination-port
 match interface input
 match flow direction
 collect routing next-hop address ipv4
 collect counter bytes
!

Create the flow monitor

!*******************************
! FLOW MONITOR
! Creates a new NetFlow cache
! Attach the flow record
! Exporter is attached to the cache
! Potential sampling configuration
!
flow monitor MONITOR-STATS
 cache timeout inactive 60
 cache timeout active 60
 cache timeout update 1
 record RECORD-STATS
!

Then apply the flow monitor on the interface to the BRs:

!
interface Ethernet0/2
 description -- TO BORDER ROUTERS --
 ip flow monitor MONITOR-STATS input
 ip flow monitor MONITOR-STATS output
!

You can now check the NetFlow cache on R2and check that we have critical applications with DSCP 0x1A (AF31) and voice flows with DSCP 0x2E (EF) (this is just and extract of the cache):


R2#sh flow monitor MONITOR-STATS cache format table
  Cache type:                               Normal
  Cache size:                                 4096
  Current entries:                             859
  High Watermark:                              882

  Flows added:                                5958
  Flows aged:                                 5099
    - Active timeout      (    60 secs)       5099
    - Inactive timeout    (    60 secs)          0
    - Event aged                                 0
    - Watermark aged                             0
    - Emergency aged                             0

IPV4 SRC ADDR    IPV4 DST ADDR    TRNS SRC PORT  TRNS DST PORT  INTF INPUT            FLOW DIRN  IP DSCP  IP PROT  ipv4 next hop addr       bytes
===============  ===============  =============  =============  ====================  =========  =======  =======  ==================  ==========

10.0.100.10      10.10.10.2               57919           1967  Et0/2                 Input      0x00          17  0.0.0.0                     80
10.0.100.10      10.10.10.2               57919           5000  Et0/2                 Input      0x00          17  0.0.0.0                   1200
10.0.100.10      10.10.10.2               53559           1967  Et0/2                 Input      0x2E          17  0.0.0.0                     80
10.0.200.10      10.10.10.2               64931           1967  Et0/2                 Input      0x2E          17  0.0.0.0                     80
10.10.1.1        10.1.9.200               10000          20000  Et0/1                 Output     0x2E          17  10.10.45.4                2640
10.10.1.1        10.1.10.200              20000          10000  Et0/1                 Output     0x2E          17  10.10.45.5                2640
10.1.10.61       10.10.3.1                 7039           7000  Et0/2                 Input      0x1A           6  10.10.12.1                 795
10.1.10.24       10.10.1.1                 2005             80  Et0/2                 Input      0x1A           6  10.10.12.1                 786
10.1.10.62       10.10.2.1                 1048             25  Et0/2                 Input      0x1A           6  10.10.12.1                 787
10.10.3.1        10.1.10.61                7000           7039  Et0/1                 Output     0x1A           6  10.10.45.5               10737
10.10.1.1        10.1.10.24                  80           2005  Et0/1                 Output     0x00           6  10.10.45.5               10737
10.10.2.1        10.1.10.62                  25           1048  Et0/1                 Output     0x00           6  10.10.45.5               10737
10.10.1.1        10.1.9.7                    80           2072  Et0/1                 Output     0x00           6  10.10.45.4               10737


{SNIP]



PfR Configuration

Presentation

In this guide we want to protect voice/video and critical applications against soft errors, brownouts and blackouts. We want to be able to track jitter, delay and loss but we also want to have a fast failover. Therefore performance measurement will be in active mode and will use jitter probes. Jitter probe configuration for all the remote sites is a painful process and Cisco Performance Routing is now using a new feature called Target Discovery to help simplifying the configuration and deployment in such cases.

A peering session is configured between the Master Controller on the central site and the Master Controllers on the remote sites. The MCs which participate in this peering will exchange the list of inside prefixes and IP SLA probe responder addresses that belong to the sites, along with the site name (configurable).


Pfr-td1.png


Provisioning

Basic configuration to establish session between MC and BR on the central site:

! This is the basic configuration required to establish session between MC and BR.
! With this basic configuration, learning is enabled, route control is in place and
! load-balancing happens on all configured external interfaces.
!
! It includes
! – Key-chain configuration for authentication.
!
! – Specification of BR’s IP Address and internal/external interface on the BR.
!   If there is a tunnel between the BRs, it should be configured as internal
!
! - Specification of the Carrier name (link-group). Link-group is used to color the exit interfaces. 
!   We will define an administrative policy for voice/video and critical application to choose SP1 
!   has the primary path and SP2 as the fallback path.
!
! - Load balancing range set to 30%. This is conservative and helps to cut down on the churn resulting 
!   from movement of traffic-classes across exits.
!
! - Disabling auto-tunnels as BRs are L2 adjacent.
! - Forcing the use of PBR
! - Specification of the probe packet number. Used to decrease the CPU utilization.
!
! Notes on the new defaults:
! - Automatic learning is enabled
! - mode route control is enabled
! - Maximum transmit utilization is 90%
! - Load-balancing is now used as the last resolver and cannot be disabled.
!
!
pfr master
 max-range-utilization percent 30
 !
 border 10.10.10.4 key-chain pfr
  interface Tunnel100 external
   link-group SP1
  interface Ethernet0/0 internal
 !
 border 10.10.10.5 key-chain pfr
  interface Tunnel200 external
   link-group SP2
  interface Ethernet0/0 internal
 !
 mode route protocol pbr
 no mode auto-tunnels
 periodic 90
 probe packets 20
 !
!


Enable NetFlow version9 Export (optional)


! Flow Exporter Definition
! Where do I want my data sent?
!
flow exporter MYEXPORTER
 destination 10.151.1.131
 source Loopback0
 transport udp 9991
 option interface-table timeout 300
 option sampler-table timeout 300
 option application-table timeout 300
!
!
! Add NetFlow export to PfR
!
pfr master
 exporter MYEXPORTER
!


Enable logging


! Following configuration is to enable logging. 
! This will print PfR related syslog messages on the console.
!
pfr master
 logging
!


Basic configuration to establish session between MC and BR on the branch sites:

!
! BRANCH1 - R9
!
key chain pfr
 key 0
  key-string cisco
!
pfr master
 logging
 !
 border 10.2.9.9 key-chain pfr
  interface Ethernet0/0 internal
  interface Tunnel100 external
   link-group SP1
  interface Tunnel200 external
   link-group SP2
 !
!
pfr border
 logging  
 local Loopback0
 master 10.2.9.9 key-chain pfr
!


!
! BRANCH2 - R10
!
key chain pfr
 key 0
  key-string cisco
!
pfr master
 logging
 !
 border 10.2.10.10 key-chain pfr
  interface Ethernet0/0 internal
  interface Tunnel100 external
   link-group SP1
  interface Tunnel200 external
   link-group SP1
 !
!
pfr border
 logging
 local Loopback0
 master 10.2.10.10 key-chain pfr
!


Enabling PfR Domain and Target Discovery

The Performance Routing (PfR) Target Discovery feature introduces a scalable solution for managing the performance of video, voice and critical applications across large Enterprise branch networks by automating the identification and configuration of IP SLA responders. To optimize media applications using voice and video traffic, PfR uses jitter, loss, and delay measurements. The IP SLA udp-jitter probe provides these measurements but requires an IP SLA responder. The initial PfR solution was to manually configure the probes for all remote sites. PfR Target Discovery uses the PfR Domain Peering framework to advertize site prefixes and to identify and advertize IP SLA responder IP addresses.


Enabling Target Discovery on R3:

!
! Target Discovery is used in a hub and spoke model where R3 is the hub and R9/R10 are the spokes.
! R3 is listening to incoming connections from the spokes.
! We want to define the IP address of the shadow router used as a responder on the hub. We also want to control the list of prefixes advertized to the spokes. Therefore we use the responder-list and inside-prefix options.
!
pfr master
 mc-peer domain 65000 head-end Loopback0
 target-discovery responder-list RESPONDER_PREFIX inside-prefixes HQ_PREFIX
!
ip prefix-list HQ_PREFIX seq 5 permit 10.10.1.0/24
ip prefix-list HQ_PREFIX seq 10 permit 10.10.2.0/24
ip prefix-list HQ_PREFIX seq 15 permit 10.10.3.0/24
ip prefix-list HQ_PREFIX seq 20 permit 10.10.4.0/24
ip prefix-list RESPONDER_PREFIX seq 5 permit 10.10.10.2/32
!


Enabling Target Discovery on R9:

!
! Target Discovery is used in a hub and spoke model where R3 is the hub and R9/R10 are the spokes.
! R9 is configured with the IP address of the hub (R3)
! To simplify the configuration on the branch offices, we don't want to manually define the target and prefix list. PfR will automatically generate what's needed. Bu default the responder address is the IP address of the LAN interface.
!
pfr master
 mc-peer domain 65000 10.10.10.3 Loopback0
 target-discovery
!

Enabling Target Discovery on R10:

!
!
! Target Discovery is used in a hub and spoke model where R3 is the hub and R9/R10 are the spokes.
! R10 is configured with the IP address of the hub (R3)
! To simplify the configuration on the branch offices, we don't want to manually define the target and prefix list. PfR will automatically generate what's needed. Bu default the responder address is the IP address of the LAN interface.
!
pfr master
 mc-peer domain 65000 10.10.10.3 Loopback0
 target-discovery
!


Learning Configuration

We would like PfR to learn and optimize traffic going to the branch offices. More specifically, of the streams that are headed to that branch, we would like PfR to optimize certain mission critical traffic based on certain policy parameters and the rest of the traffic based on a different set of parameters.

We also want to apply policies not on a global basis but for specific application groups. The choice here is to define application groups by filtering on DSCP values.

We assume that traffic is already marked when it enters the BRs on all sites. We can therefore define learning based on DSCP values while filtering based on the branch prefix. The filtering (based on the prefix) will make sure that PfR only learns traffic destined to the remote branch offices, while the different access-lists (based on DSCP values) help categorize the interesting traffic into different buckets like critical, best effort, etc., which can then be associated with different optimization policies.

Define learning parameters:


! By default now:
!
! - Automatic learning is enabled
!
! - Continuous learn cycle, each 1 minute duration 
!   periodic-interval = 0 (forever)
!   monitor-period = 1 (minutes)
!
! - traffic-class sorting based on ‘throughput’ at the end of each learning cycle.
!
! - Anything traffic that doesn’t match the learn list will be learned under global learn and will be optimized using default policy. 
!
pfr master
 !
 !
 learn
  ! - Define 3 groups of applications
  ! - disable global learn.
  ! - Sort the traffic-class based on ‘throughput’ at the end of each learning cycle. 
  !
  throughput
  !
  ! For large scale deployment you may want to define the periodic interval from 0 (infinite) to 1 minute. 
  ! This allows the MC to have time to learn the new Traffic Classes and avoid having possible overlapping learning periods.
  ! In this solution guide we will keep the default configuration
  !
  ! periodic-interval 1
  !
  ! We do not want to use global learning. 
  ! Learn-list has been optimized for learning and high number of TCs. 
  ! We also want to use this for all Traffic Classes including the best effort ones.
  ! Therefore we disable global learning.
  !
  traffic-class filter access-list DENY_GLOBAL_LEARN_LIST
  !
  !-------------------------------------------------------------------
  ! Service Class for Voice and Video traffic
  ! Control the max number of new TCs per learning period with 'count'
  ! Control the max total number of TCs in this learn-list with 'max'
  ! Define the Aggregation Mask
  !
  list seq 10 refname LEARN_VOICE_VIDEO
   traffic-class access-list VOICE_VIDEO filter BRANCH_PREFIX
   aggregation-type prefix-length 24
   count 2000 max 10000
   throughput
  !
  !-------------------------------------------------------------------
  ! Service Class for Business applications
  ! Control the max number of new TCs per learning period with 'count'
  ! Control the max total number of TCs in this learn-list with 'max'
  ! Define the Aggregation Mask
  !
  list seq 20 refname LEARN_CRITICAL
   traffic-class access-list CRITICAL filter BRANCH_PREFIX
   aggregation-type prefix-length 24
   count 2000 max 10000
   throughput
  !
  !-------------------------------------------------------------------
  ! Service Class for Best Effort applications
  ! Control the max number of new TCs per learning period with 'count'
  ! Control the max total number of TCs in this learn-list with 'max'
  ! Define the Aggregation Mask
  !
  list seq 30 refname LEARN_BEST_EFFORT
   traffic-class access-list BEST_EFFORT filter BRANCH_PREFIX
   aggregation-type prefix-length 24
   count 2000 max 10000
   throughput
 !
!
!---------------------------------------------------------------------
! ACL to deny global learning
!
ip access-list extended DENY_GLOBAL_LEARN_LIST
 deny   ip any any
!
!---------------------------------------------------------------------
! Voice and Video traffic classified based on DSCP EF, AF41 and CS4 (Tandberg)
!
ip access-list extended VOICE_VIDEO
 permit ip any any dscp ef
 permit ip any any dscp af41
 permit ip any any dscp cs4
!
!---------------------------------------------------------------------
! Business application classified based on DSCP AF31
!
ip access-list extended CRITICAL
 permit ip any any dscp af31
!
!---------------------------------------------------------------------
! Everything else is best effort
!
ip access-list extended BEST_EFFORT
 permit ip any any dscp default
!
!
!---------------------------------------------------------------------
! Filter to track traffic going to the branch only.
!
ip prefix-list BRANCH_PREFIX seq 10 permit 10.1.0.0/16
!
!

Important notes:

  • A Traffic Class is an aggregation of individual flows based on destination prefix, DSCP and application name. Defining the appropriate aggregation mask length is key for TD operation to work properly and generates jitter probes to the remote sites.
  • Aggregation mask should match the mask length advertized by Target Discovery (TD).
  • In this case all branch sites advertized a /24 prefix. Therefore the aggregation mask is set to 24 (this is the default and won't appear in the final configuration.


Policy Configuration

Monitoring Modes

We have 3 groups of applications that require different monitoring modes. See Monitoring Modes for more information.

For best effort applications, running passive mode is enough because traffic is mainly based on TCP. For voice, traffic is based on UDP (RTP) and we may want to get additional metrics like delay, jitter or even MOS. So we have to go with an active mode. For critical application we may also want to run in active mode.

To acquire jitter metrics, the router will need to inject instrumented synthetic traffic and derive the required information from there. With TCP traffic, we cannot make passive jitter measurements---only latency, loss and throughput. PfR makes use of the IOS IPSLA feature to generate probes and collect information about the state of the network.

In the case of jitter measurements, IPSLA requires that the probe destination needs to be an IPSLA responder. The IPSLA responder function is available on IOS devices as well as Cisco Telepresence units. The IPSLA responder is able to make accurate latency time stamping and has the vital ability of factoring out the processing time taken within the responder between reception of the probe and the generation of the probe reply.

Realistically, having access to a IPSLA responder along the path of a data traffic is usually only within the scope of private IP networks.

Note that if a traffic-class is monitored under active mode (which is completely active measurement based), passive measurements are disabled for that traffic-class. Notably, the throughput information is unavailable. PfR offers an hybrid monitoring mode called ‘active throughput’ that will trigger OOP based on the active measurements, but throughput information is also collected. The CLI for this command is ‘mode monitor active throughput’ and is used for the critical applications in this lab.


Defining Advanced Policies per Group

PfR-maps are somewhat similar to route-maps in functionality in that a pfr-map will ‘match’ traffic and then apply (or in other words ‘set’) a form of policy to that traffic. Now that the network is able to identify that traffic (via the ACLs and the learn-lists) we can configure the pfr-map entries. So far, PfR was optimizing all traffic based on the inherited global policies, which emphasized load sharing. Also, measurements were done using the default monitor mode of ‘both’ (meaning a combination of ‘passive’ and ‘active’) for all traffic.

VIDEO group:

  • Our voice/video traffic is matching DSCP EF marked flows
  • Latency, jitter and loss would be more important. Another possibility (rather than specifying delay, jitter and loss individually) is that we could have represented our performance target in terms of Mean Opinion Score (MOS), which is a composite metric based on loss, jitter, latency and the specific codec being used. Specifying the target in terms of MOS is much simpler and the reference being used is voice quality (for example, MOS 4 is toll quality voice service).
  • Measurement mode should be ‘fast’ to be able to track delay and loss quickly and more accurately. So, jitter probe is needed and an explicit jitter configuration per branch would be required. Because we use Target Discovery, there is no need for jitter probe configuration anymore.

CRITICAL group:

  • Our critical traffic class is matching DSCP AF31 marked flows.
  • Latency and loss would be more important.
  • Measurement mode should be ‘active throughput’ to be able to track delay and loss as well as getting the bandwidth per traffic class. So, jitter probe is needed and an explicit jitter configuration per branch would be required. Because we use Target Discovery, there is no need for jitter probe configuration anymore.

BE group:

  • Best effort traffic on the other hand can still be optimized for load-balancing.
  • Measurement mode can be passive.


Configuration

Policy configuration for controlling applications:

!-------------------------------------------------------------------------------
! Policies for Voice and Video
!
! – match command is to specify that this policy should be applied
!   to all the traffic-classes learned under list LEARN_VOICE_VIDEO
!
! - monitor mode is set to fast. This means probe all external interfaces
!   all the time. When Out-of-Policy condition is detected on the current 
!   exit results on alternate exit is available for quick decision. In other 
!   modes alternate exits are probed only when current link is determined to 
!   be OOP. The fast mode helps in switching the path quickly when the 
!   problem is detected.
!
! - Re-evaluate exit every 90 sec (periodic 90)
!
! – delay threshold is configured as 150 msec. The delay measured by PfR is Round-Trip-Time.
! - loss threshold is configured as 5%
! - jitter threshold is configured as 30 ms
!
! – Probe frequency is set to 8 seconds and can be changed to a lesser value
!   if the application being controlled is critical.
!
! - Probes are automatically configured and generated by Target Discovery
!
!-------------------------------------------------------------------------------
!
pfr-map MYMAP 10
 match PfR learn list LEARN_VOICE_VIDEO
 set periodic 90
 set delay threshold 200
 set loss threshold 50000
 set jitter threshold 30
 set mode monitor fast
 set resolve delay priority 1 variance 5
 set resolve loss priority 2 variance 5
 set resolve jitter priority 3 variance 5
 set probe frequency 8
 set link-group SP1 fallback SP2
!
!
!-------------------------------------------------------------------------------
! Policies for Critical Data
!
! – match command is to specify that this policy should be applied
!   to all the traffic-classes learned under list LEARN_CRITICAL
!
! - monitor mode is set to fast. This means probe all external interfaces
!   all the time. When Out-of-Policy condition is detected on the current 
!   exit results on alternate exit is available for quick decision. In other 
!   modes alternate exits are probed only when current link is determined to 
!   be OOP. The fast mode helps in switching the path quickly when the 
!   problem is detected.
!
! - Re-evaluate exit every 90 sec (periodic 90)
!
! – delay threshold is configured as 100 msec. The delay measured by PfR is Round-Trip-Time.
! - loss threshold is configured as 5%
!
! – Probe frequency is set to 8 seconds and can be changed to a lesser value
!   if the application being controlled is critical.
!
! - Probes are automatically configured and generated by Target Discovery
!
!-------------------------------------------------------------------------------
!
pfr-map MYMAP 20
 match pfr learn list LEARN_CRITICAL
 set periodic 90
 set delay threshold 100
 set mode monitor fast
 set resolve delay priority 1 variance 20
 set resolve loss priority 5 variance 10
 set probe frequency 8
 set link-group SP1 fallback SP2
!
!
!-------------------------------------------------------------------------------
! Policies for Best Effort
!
! – match command is to specify that this policy should be applied
!   to all the traffic-classes learned under list LEARN_BEST_EFFORT
!
! - monitor mode is set to both. This is the default mode. Monitoring is passive and 
!    echo probes are used to help finding a new exit when a TC is OOP 
!
! - Default policies used: link utilization first then load-balancing
!
!-------------------------------------------------------------------------------
!
pfr-map MYMAP 30
 match pfr learn list LEARN_BEST_EFFORT
 set periodic 90
 set mode monitor both
!


Now that the policies are defined using pfr-map, we need to apply them on the PfR MC for them to take effect. This is done through the ‘policy-rules’ command under ‘pfr master’ global mode:

!
pfr master
 policy-rules MYMAP
!


Check Master Controllers

Check Status

Verify the Border Routers, verify the parameters used (default and configured) and check the learn-list.


R3#sh pfr master
OER state: ENABLED and ACTIVE
  Conn Status: SUCCESS, PORT: 3949
  Version: 3.3
  Number of Border routers: 2
  Number of Exits: 2
  Number of monitored prefixes: 8 (max 5000)
  Max prefixes: total 5000 learn 2500
  Prefix count: total 8, learn 6, cfg 0
  PBR Requirements met
  Nbar Status: Inactive

Border           Status                UP/DOWN             AuthFail  Version  DOWN Reason
10.10.10.4       ACTIVE                UP       00:38:01          0  3.3
10.10.10.5       ACTIVE                UP       00:38:00          0  3.3

Global Settings:
  max-range-utilization percent 30 recv 0
  rsvp post-dial-delay 0 signaling-retries 1
  mode route metric bgp local-pref 5000
  mode route metric static tag 5000
  mode route protocol pbr
  trace probe delay 1000
  logging 
  exit holddown time 60 secs, time remaining 0

Default Policy Settings:
  backoff 90 900 90
  delay relative 50
  holddown 90
  periodic 90
  probe frequency 56
  number of jitter probe packets 20
  mode route control 
  mode monitor both
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  trigger-log percentage 30

Learn Settings:
  current state : STARTED
  time remaining in current state : 110 seconds
  throughput
  no delay
  no inside bgp
  traffic-class filter access-list DENY_GLOBAL_LEARN_LIST
  monitor-period 1
  periodic-interval 0
  aggregation-type prefix-length 24
  prefixes 100 appls 100
  expire after time 720

  Learn-List seq 10 refname LEARN_VOICE_VIDEO
    Configuration:
     Traffic-Class Access-list: VOICE_VIDEO
     Filter: BRANCH_PREFIX
     Aggregation-type: prefix-length 24
     Learn type: throughput
     Session count: 2000 Max count: 10000
     Policies assigned: 10
     Status: ACTIVE
    Stats:
     Traffic-Class Count: 2
  Learn-List seq 20 refname LEARN_CRITICAL
    Configuration:
     Traffic-Class Access-list: CRITICAL
     Filter: BRANCH_PREFIX
     Aggregation-type: prefix-length 24
     Learn type: throughput
     Session count: 2000 Max count: 10000
     Policies assigned: 20
     Status: ACTIVE
    Stats:
     Traffic-Class Count: 2
  Learn-List seq 30 refname LEARN_BEST_EFFORT
    Configuration:
     Traffic-Class Access-list: BEST_EFFORT
     Filter: BRANCH_PREFIX
     Aggregation-type: prefix-length 24
     Learn type: throughput
     Session count: 2000 Max count: 10000
     Policies assigned: 30
     Status: ACTIVE
    Stats:
     Traffic-Class Count: 2
R3#

What to check:

  • Both Border Routers are up and running
  • Check to make sure that PBR Requirements are met
  • Number of automatically learned applications – 2 in this case for Voice, 2 for Critical and 2 for best effort. We have 2 branch offices and aggregation mask is per branch.
  • All default policy settings are displayed
  • Learn is started (current state : STARTED)
  • Learn-list status is ACTIVE and learn-type is throughput
  • Make sure that all the configured learn lists show up
  • For each learn-list, check the Traffic Class access-list (if configured), prefix-list (if configured) and the policy number associated (which is the pfr-map it refers to).
  • The Traffic Class count is also displayed which allows you to check whether the learning process worked well.


Check the policies:


R3#sh pfr master policy
Default Policy Settings:
  backoff 90 900 90
  delay relative 50
  holddown 90
  periodic 90
  probe frequency 56
  number of jitter probe packets 20
  mode route control 
  mode monitor both
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  trigger-log percentage 30
oer-map MYMAP 10
  sequence no. 8444249301975040, provider id 1, provider priority 30
    host priority 0, policy priority 10, Session id 0
  match oer learn list LEARN_VOICE_VIDEO
  backoff 90 900 90
 *delay threshold 150
  holddown 90
 *periodic 90
 *probe frequency 8
  number of jitter probe packets 20
  mode route control 
 *mode monitor fast
 *loss threshold 50000
 *jitter threshold 30
  mos threshold 3.60 percent 30
  unreachable relative 50
  next-hop not set
  forwarding interface not set
  trigger-log percentage 30
 *resolve delay priority 1 variance 5
 *resolve loss priority 2 variance 5
 *resolve jitter priority 3 variance 5
 *link-group SP1 fallback SP2
oer-map MYMAP 20
  sequence no. 8444249302630400, provider id 1, provider priority 30
    host priority 0, policy priority 20, Session id 0
  match oer learn list LEARN_CRITICAL
  backoff 90 900 90
 *delay threshold 100
  holddown 90
 *periodic 90
 *probe frequency 8
  number of jitter probe packets 20
  mode route control 
 *mode monitor fast
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  next-hop not set
  forwarding interface not set
  trigger-log percentage 30
 *resolve delay priority 1 variance 20
 *resolve loss priority 5 variance 10
 *link-group SP1 fallback SP2
oer-map MYMAP 30
  sequence no. 8444249303285760, provider id 1, provider priority 30
    host priority 0, policy priority 30, Session id 0
  match oer learn list LEARN_BEST_EFFORT
  backoff 90 900 90
  delay relative 50
  holddown 90
 *periodic 90
  probe frequency 56
  number of jitter probe packets 20
  mode route control 
 *mode monitor both
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  next-hop not set
  forwarding interface not set
  trigger-log percentage 30

* Overrides Default Policy Setting
R3#


You can also check the policy associated to a specific Service Class. For LEARN_VOICE_VIDEO:


R3#sh pfr master policy 10
* Overrides Default Policy Setting
oer-map MYMAP 10
  sequence no. 8444249301975040, provider id 1, provider priority 30
    host priority 0, policy priority 10, Session id 0
  match oer learn list LEARN_VOICE_VIDEO
  backoff 90 900 90
 *delay threshold 150
  holddown 90
 *periodic 90
 *probe frequency 8
  number of jitter probe packets 20
  mode route control 
 *mode monitor fast
 *loss threshold 50000
 *jitter threshold 30
  mos threshold 3.60 percent 30
  unreachable relative 50
  next-hop not set
  forwarding interface not set
  trigger-log percentage 30
 *resolve delay priority 1 variance 5
 *resolve loss priority 2 variance 5
 *resolve jitter priority 3 variance 5
 *link-group SP1 fallback SP2
R3#


Notes:

  • Default policies that will be applied are somewhat similar to the concept of default-class in QoS. All Traffic that does not match learn-list falls under global policies. Load balancing is enabled by default for this kind of traffic. In this solution guide we have disable global learning.
  • Then note the additional section under ‘pfr-map MYMAP 10’, ‘pfr-map MYMAP 20’ and ‘pfr-map MYMAP 30’ (just like route-maps, we can have multiple stanzas of match critera and policies).
  • Note how the pfr-map inherits the policy settings from default for parameters that were not explicitly configured under the pfr-map entry.



Check Target Discovery and Peering

On the central site:


R3#sh pfr master target-discovery 
PfR Target-Discovery Services
 Mode: Static  Domain: 65000
 Responder list: RESPONDER_PREFIX  Inside-prefixes list: HQ_PREFIX
 SvcRtg: client-handle: 1  sub-handle: 1  pub-seq: 1

PfR Target-Discovery Database (local)

 Local-ID: 10.10.10.3        Desc: R3
   Target-list: 10.10.10.2
   Prefix-list: 10.10.4.0/24, 10.10.3.0/24, 10.10.2.0/24, 10.10.1.0/24

PfR Target-Discovery Database (remote)

 MC-peer: 10.2.10.10         Desc: R10
   Target-list: 10.1.10.254
   Prefix-list: 10.1.10.0/24

 MC-peer: 10.2.9.9           Desc: R9
   Target-list: 10.1.9.254
   Prefix-list: 10.1.9.0/24


R3#

What to Check:

  • You get the list of all remote sites
  • You get the list of destination prefixes per remote site
  • You also get the probe target addresses per remote site (there could be multiple responders within a given site)


You can display the state of active probes on the MC.
Remember that IP SLA probes are generated by the BRs (R4 and R5), which report back the results to the MC.


R3#sh pfr master active-probes target-discovery 
PfR Master Controller active-probes (TD)
Border	= Border Roter running this probe
MC-Peer	= Remote MC associated with this target
Type	= Probe Type
Target	= Target Address
TPort	= Target Port
N - Not applicable

Destination Site Peer Addresses:

MC-Peer           Targets
10.2.10.10        10.1.10.254
10.2.9.9          10.1.9.254

The following Probes are running:

Border          Idx  State     MC-Peer            Type     Target           TPort
10.10.10.4       8   TD-Actv   10.2.10.10         jitter   10.1.10.254      5000
10.10.10.5      10   TD-Actv   10.2.10.10         jitter   10.1.10.254      5000
10.10.10.5      10   TD-Actv   10.2.10.10         jitter   10.1.10.254      5000
10.10.10.4       8   TD-Actv   10.2.10.10         jitter   10.1.10.254      5000
10.10.10.4       8   TD-Actv   10.2.10.10         jitter   10.1.10.254      5000
10.10.10.5      10   TD-Actv   10.2.10.10         jitter   10.1.10.254      5000
10.10.10.4       8   TD-Actv   10.2.9.9           jitter   10.1.9.254       5000
10.10.10.5      10   TD-Actv   10.2.9.9           jitter   10.1.9.254       5000
10.10.10.4       8   TD-Actv   10.2.9.9           jitter   10.1.9.254       5000
10.10.10.5      10   TD-Actv   10.2.9.9           jitter   10.1.9.254       5000
10.10.10.5      10   TD-Actv   10.2.9.9           jitter   10.1.9.254       5000
10.10.10.4       8   TD-Actv   10.2.9.9           jitter   10.1.9.254       5000


R3#


Check Target Discovery on the branch site R9:


R9#sh pfr master target-discovery 
PfR Target-Discovery Services
 Mode: Dynamic  Domain: 65000
 SvcRtg: client-handle: 1  sub-handle: 1  pub-seq: 1

PfR Target-Discovery Database (local)

 Local-ID: 10.2.9.9          Desc: R9
   Target-list: 10.1.9.254
   Prefix-list: 10.1.9.0/24

PfR Target-Discovery Database (remote)

 MC-peer: 10.10.10.3         Desc: R3
   Target-list: 10.10.10.2
   Prefix-list: 10.10.4.0/24, 10.10.3.0/24, 10.10.2.0/24, 10.10.1.0/24

 MC-peer: 10.2.10.10         Desc: R10
   Target-list: 10.1.10.254
   Prefix-list: 10.1.10.0/24

R9#

Note:

  • R9 has only one peer which is the hub R3
  • But R9 learns all prefixes from all sites, including the other spokes. In this example R9 has inside prefixes from R10.


You can display the state of active probes on the MC.
Remember that IP SLA probes are generated by the BRs, which report back the results to the MC.


R9#sh pfr master active-probes target-discovery 
PfR Master Controller active-probes (TD)
Border	= Border Roter running this probe
MC-Peer	= Remote MC associated with this target
Type	= Probe Type
Target	= Target Address
TPort	= Target Port
N - Not applicable

Destination Site Peer Addresses:

MC-Peer           Targets
10.10.10.3        10.10.10.2
10.2.10.10        10.1.10.254

The following Probes are running:

Border          Idx  State     MC-Peer            Type     Target           TPort
10.2.9.9         9   TD-Actv   10.10.10.3         jitter   10.10.10.2       5000
10.2.9.9        10   TD-Actv   10.10.10.3         jitter   10.10.10.2       5000
10.2.9.9         9   TD-Actv   10.10.10.3         jitter   10.10.10.2       5000
10.2.9.9        10   TD-Actv   10.10.10.3         jitter   10.10.10.2       5000
10.2.9.9        10   TD-Actv   10.2.10.10         jitter   10.1.10.254      5000
10.2.9.9         9   TD-Actv   10.2.10.10         jitter   10.1.10.254      5000


R9#

Note:

  • Multiple probe types generated: echo for critical applications (no need for jitter information) and jitter probes (voice/video traffic).
  • Target Discovery will only configure and generate jitter probes for fast mode.
  • The probe packets will automatically take on the DSCP properties of the traffic class. In the case that the traffic class type has multiple DSCP settings (which is very likely when the traffic class is learned rather than explicitly specified), a unique probe for each DSCP value will be generated.
    • Voice:
      • DSCP EF (46, 0x2E)
      • TOS = 0xB8, 184
    • Business:
      • DSCP AF31 (26, 0x1A)
      • TOS = 0x68, 104
  • Note that there are probes from R9 to R10. This is because we have voice traffic between spokes. When there is no traffic, or if voice traffic stops, then probe generation between spokes will also stop.


Notes specific to Target Discovery probes

  • Traffic classes going to the same remote site share probe statistics
  • Only one set of probes will be run on each BR even thought multiple applications to the same site are being monitored by PfR
  • Probes to TD targets are always of type jitter
  • If there are multiple targets advertised by a remote site, one probes to all targets will be run simultaneously


Verify Traffic Class and Statistics

As soon as you see:

R3#
*Sep 20 20:24:43.515: %OER_MC-5-NOTICE: Prefix Learning WRITING DATA
*Sep 20 20:24:43.589: %OER_MC-5-NOTICE: Prefix Learning STARTED
R3#


You should be able to see the traffic classes on the Master Controller. You will need a few cycles before having all applications in INPOLICY state.


Check if applications are automatically learnt under each learn-list


R3#sh pfr master learn list

 Learn-List seq 10 refname LEARN_VOICE_VIDEO
   Configuration:
    Traffic-Class Access-list: VOICE_VIDEO
    Filter: BRANCH_PREFIX
    Aggregation-type: prefix-length 24
    Learn type: throughput
    Session count: 2000 Max count: 10000
    Policies assigned: 10
    Status: ACTIVE
   Stats:
    Traffic-Class Count: 2
    Traffic-Class Learned:
     Appl Prefix 10.1.10.0/24 ef   256
     Appl Prefix 10.1.9.0/24 ef   256
 Learn-List seq 20 refname LEARN_CRITICAL
   Configuration:
    Traffic-Class Access-list: CRITICAL
    Filter: BRANCH_PREFIX
    Aggregation-type: prefix-length 24
    Learn type: throughput
    Session count: 2000 Max count: 10000
    Policies assigned: 20
    Status: ACTIVE
   Stats:
    Traffic-Class Count: 2
    Traffic-Class Learned:
     Appl Prefix 10.1.9.0/24 af31 256
     Appl Prefix 10.1.10.0/24 af31 256
 Learn-List seq 30 refname LEARN_BEST_EFFORT
   Configuration:
    Traffic-Class Access-list: BEST_EFFORT
    Filter: BRANCH_PREFIX
    Aggregation-type: prefix-length 24
    Learn type: throughput
    Session count: 2000 Max count: 10000
    Policies assigned: 30
    Status: ACTIVE
   Stats:
    Traffic-Class Count: 2
    Traffic-Class Learned:
     Appl Prefix 10.1.9.0/24 defa 256
     Appl Prefix 10.1.10.0/24 defa 256
R3#


Verify learnt Traffic Classes

On the Master Controller, you have all the Traffic Classes (learnt as well as statistics): Use show pfr master traffic-class to see the state of the application:


R3#sh pfr master traffic-class 
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------
10.1.9.0/24               N defa  256           N           N 0.0.0.0/0         
                          INPOLICY       23        10.10.10.5 Tu200           PBR     
               U        U        0        0        0        0      179        0
               U       72        0        0        0        U        0        0

10.1.9.0/24               N af31  256           N           N 0.0.0.0/0         
                          INPOLICY       @6        10.10.10.4 Tu100           PBR     
              68       68        0        0        0        0       88       19
              55       54        0        0        3        0        0        0

10.1.9.0/24               N   ef  256           N           N 0.0.0.0/0         
                          INPOLICY       @7        10.10.10.4 Tu100           PBR     
               U        U        0        0        0        0        1        1
              58       58        0        0        4        0        0        0

10.1.10.0/24              N defa  256           N           N 0.0.0.0/0         
                          INPOLICY       24        10.10.10.5 Tu200           PBR     
               U        U        0        0        0        0      178        0
               U       65        0        0        0        U        0        0

10.1.10.0/24              N af31  256           N           N 0.0.0.0/0         
                          INPOLICY      @68        10.10.10.4 Tu100           PBR     
              67       67        0        0        0        0       90       19
              67       64        0        0        6        0        0        0

10.1.10.0/24              N   ef  256           N           N 0.0.0.0/0         
                          INPOLICY       @2        10.10.10.4 Tu100           PBR     
               U        U        0        0        0        0        1        1
              66       64        0        0        6        0        0        0

R3#

Notes:

  • Note the DSCP values under the DSCP field column.
  • The TCs belonging to CRITICAL and VOICE_VIDEO learn-list are being forced out R4 as it has a delay within the defined threshold to the remote branch and because the preferred path is SP1 (note that R5 would be a valid choice too without the link-group policy).
  • The TCs belonging to BEST_EFFORT are supposed to be load-balanced across both Border Routers and exit links.
  • Note the path enforcement used.


You can also display more details to check the probe used and latest statistics per Traffic Class:

R3#sh pfr master traffic-class detail
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------

[SNIP]

--------------------------------------------------------------------------------

Prefix: 10.1.9.0/24  Protocol: 256  Port: [0, 0] [0, 0] DSCP: ef  
   State: INPOLICY    Time Remaining: @55     
   Policy: MYMAP 10

   Most recent data per exit
   Border          Interface           PasSDly    PasLDly    ActSDly    ActLDly
  *10.10.10.4      Tu100                     0          0         55         58
   10.10.10.5      Tu200                     0          0         66         66

   Most recent voice data per exit
   Border          Interface         ActSJit  ActPMOS  ActSLos  ActLLos
  *10.10.10.4      Tu100                   3        0        0        0
   10.10.10.5      Tu200                   3        0        0        0

   Latest Active Stats on Current Exit:
   Type     Target          TPort Attem Comps    DSum     Min     Max     Dly
   jitter   10.1.9.254       5000     1    20    1075      45     219      53
   jitter   10.1.9.254       5000     1    20    1099      45     219      54
   jitter   10.1.9.254       5000     1    20    1222      45     219      61
   jitter   10.1.9.254       5000     1    20    1112      45     219      55
   jitter   10.1.9.254       5000     1    20    1086      45     219      54


   Latest Active Voice Stats on Current Exit:
   Type     Target          TPort     Codec Attem Comps  JitSum      MOS
   jitter   10.1.9.254      5000      g729a     1    20      51     4.06
   jitter   10.1.9.254      5000      g729a     1    20      43     4.06
   jitter   10.1.9.254      5000      g729a     1    20     111     4.06
   jitter   10.1.9.254      5000      g729a     1    20      57     4.06
   jitter   10.1.9.254      5000      g729a     1    20     103     4.06

Prefix performance history records
 Current index 38, S_avg interval(min) 5, L_avg interval(min) 60

Age       Border          Interface       OOP/RteChg Reasons                  
Pas: DSum  Samples  DAvg  PktLoss  Unreach   Ebytes   Ibytes     Pkts    Flows
Act: Dsum Attempts  DAvg    Comps  Unreach   Jitter LoMOSCnt   MOSCnt
00:00:06  10.10.10.4      Tu100                                               
Pas:    0        0     0        0        0        0        0        0        0
Act: 1075        1    53       20        0        2        0        1        0
00:00:14  10.10.10.4      Tu100                                               
Pas:    0        0     0        0        0        0        0        0        0
Act: 1099        1    54       20        0        2        0        1        0
00:00:22  10.10.10.4      Tu100                                               
Pas:    0        0     0        0        0     1320     1364       61        3


[SNIP]

R3#


A closer look at the results You can also have all information related to a specific traffic class with the following command:


R3#sh pfr master traffic-class performance ip any 10.1.9.0/24 dscp ef
==============================================================================================

Traffic-class:
 Destination Prefix : 10.1.9.0/24             Source Prefix    : 0.0.0.0/0
 Destination Port   : N                       Source Port      : N
 DSCP               : ef                      Protocol         : 256
 Application Name:  : N/A

 General:
   Control State                   : Controlled using PBR
   Traffic-class status            : INPOLICY
   Current Exit                    : BR 10.10.10.4 interface Tu100, Tie breaker was None
   Time on current exit            : 0d 0:12:40
   Time remaining in current state : @36 seconds
   Last uncontrol reason           : Target-Discovery probe parameter data changed
   Time since last uncontrol       : 0d 0:13:10
   Traffic-class type              : Learned
   Target-Discovery origin         : 10.2.9.9
   Improper config                 : None

 Last Out-of-Policy event:
   No Out-of-Policy Event

 Average Passive Performance Current Exit: (Average for last 5 minutes)
   Unreachable            : 0% -- Threshold: 50%
   Delay                  : 0 msecs -- Threshold: 150 msecs
   Loss                   : 0 -- Threshold: 50000 
   Egress BW              : 1 kbps
   Ingress BW             : 1 kbps
   Time since last update : 0d 0:0:0

 Average Active Performance Current Exit: (Average for last 5 minutes)
   Unreachable            : 0% -- Threshold: 50%
   Delay                  : 56 msec -- Threshold: 150 msec
   Loss                   : 0 -- Threshold: 50000 
   Jitter                 : 396 msec -- Threshold: 3000 msec

 Last Resolver Decision:
   BR              Interface    Status       Reason       Performance Threshold
   --------------- ------------ ------------ ------------ ----------- ---------
   10.10.10.5      Tu200        Eliminated   Link Group   N/A          N/A         
   10.10.10.4      Tu100        Best Exit    Unreachable  N/A          N/A         
          
==============================================================================================
R3#


Monitoring Load Balancing.

Traffic other than voice/video and critical application is load balanced across all external interfaces. To visualize the accuracy of the load balancing, the following command can be used:


R3#sh pfr master exits
==============================================================================================
PfR Master Controller Exits:

General Info:
=============
  E - External
  I - Internal
  N/A - Not Applicable
                                                                                           Up/
   ID Name         Border          Interface   ifIdx IP Address      Mask Policy      Type Down
  --- ------------ --------------- ----------- ----- --------------- ---- ----------- ---- ----
    2              10.10.10.4      Tu100           8 10.0.100.4        24 Util          E  UP
    1              10.10.10.5      Tu200          10 10.0.200.5        24 Util          E  UP

Global Exit Policy:
===================
            Cost:      In Policy

Exits Performance:
==================
                   Egress                                            Ingress
    ---------------------------------------------------- ------------------------------------
 ID Capacity  MaxUtil    Usage   %      RSVP POOL    OOP Capacity  MaxUtil    Usage   %  OOP
 --- -------- -------- -------- --- -------------- ----- -------- -------- -------- --- -----
  2     1000      900      406  40            N/A   Util     1000     1000       46   4  N/A
  1     1000      900      219  21            N/A   Util     1000     1000       44   4  N/A

TC and BW Distribution:
=======================
                       # of TCs                  BW (kbps)            Probe   Active
     Name/ID   Current Controlled InPolicy    Controlled       Total  Failed  Unreach
                                                                     (count) (fpm)
        ----   ----------------------------   ----------------------  ------  --------
           2        5          5        5          361          406      878         0
           1        1          1        1          184          219      258         0

Exit Related TC Stats:
======================
                                  Priority
                             highest       nth
                            ------------------
  Number of TCs with range:        0         0
   Number of TCs with util:        0         0
   Number of TCs with cost:        0         0

       Total number of TCs:        8
R3#

Notes:

  • You can check the bandwidth usage on all external interfaces
  • You can also check the number of TCs on each link.



Check Border Routers (BRs)

Active Probing

Check Active probing status on Border Router R4:


R4#sh pfr border active-probes
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP
jitter   10.1.9.254       5000 10.0.100.4      Tu100               227    4540
104
jitter   10.1.9.254       5000 10.0.100.4      Tu100               227    4540
184
jitter   10.1.10.254      5000 10.0.100.4      Tu100               228    4560
184
jitter   10.1.10.254      5000 10.0.100.4      Tu100               228    4560
104


R4#


Check Active probing status on Border Router R5:


R5#sh pfr border active-probes
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP
jitter   10.1.9.254       5000 10.0.200.5      Tu200                 2      40
0
jitter   10.1.9.254       5000 10.0.200.5      Tu200               230    4600
104
jitter   10.1.9.254       5000 10.0.200.5      Tu200               230    4600
184
jitter   10.1.10.254      5000 10.0.200.5      Tu200               231    4620
184
jitter   10.1.10.254      5000 10.0.200.5      Tu200               231    4620
104


R5#



You can check the IP SLA configuration:


R4#sh ip sla configuration 
IP SLAs Infrastructure Engine-III
Entry number: 23
Owner: Optimized Edge Routing (OER)
Tag: 
Operation timeout (milliseconds): 4000
Type of operation to perform: udp-jitter
Target address/Source address: 10.1.9.254/10.0.100.4
Target port/Source port: 5000/0
Type Of Service parameter: 0x68
Codec Type: g729a
Codec Number Of Packets: 20
Codec Packet Size: 32
Codec Interval (milliseconds): 20
Advantage Factor: 0
Verify data: No
Vrf Name: 
Control Packets: enabled
Schedule:
   Operation frequency (seconds): 8  (not considered if randomly scheduled)
   Next Scheduled Start Time: Start Time already passed
   Group Scheduled : FALSE
   Randomly Scheduled : FALSE
   Life (seconds): Forever
   Entry Ageout (seconds): never
   Recurring (Starting Everyday): FALSE
   Status of entry (SNMP RowStatus): Active
Threshold (milliseconds): 4000
Distribution Statistics:
   Number of statistic hours kept: 2
   Number of statistic distribution buckets kept: 1
   Statistic distribution interval (milliseconds): 20
Enhanced History:
Percentile:

[SNIP]
          
R4#


Notes:

  • Each probe has its own entry number.
  • These probes are managed by PfR (means there is no configuration done directly on the router)
  • You can check the probe type (echo or jitter)
  • For jitter you can also check the codec used, as well as packet size, interval, etc


Detailed statistics can be displayed using the following command:


R4#sh ip sla statistics 
IPSLAs Latest Operation Statistics

IPSLA operation id: 23
Type of operation: udp-jitter
	Latest RTT: 53 milliseconds
Latest operation start time: 16:07:28 CET Thu Nov 7 2013
Latest operation return code: OK
RTT Values:
	Number Of RTT: 20		RTT Min/Avg/Max: 46/53/61 milliseconds
Latency one-way time:
	Number of Latency one-way Samples: 20
	Source to Destination Latency one way Min/Avg/Max: 46/51/56 milliseconds
	Destination to Source Latency one way Min/Avg/Max: 0/1/7 milliseconds
Jitter Time:
	Number of SD Jitter Samples: 19
	Number of DS Jitter Samples: 19
	Source to Destination Jitter Min/Avg/Max: 0/3/10 milliseconds
	Destination to Source Jitter Min/Avg/Max: 0/2/7 milliseconds
Over Threshold:
	Number Of RTT Over Threshold: 0 (0%)
Packet Loss Values:
	Loss Source to Destination: 0
	Source to Destination Loss Periods Number: 0
        Source to Destination Loss Period Length Min/Max: 0/0
	Source to Destination Inter Loss Period Length Min/Max: 0/0
	Loss Destination to Source: 0
	Destination to Source Loss Periods Number: 0
	Destination to Source Loss Period Length Min/Max: 0/0
	Destination to Source Inter Loss Period Length Min/Max: 0/0
	Out Of Sequence: 0	Tail Drop: 0
	Packet Late Arrival: 0	Packet Skipped: 0
Voice Score Values:
	Calculated Planning Impairment Factor (ICPIF): 11
	MOS score: 4.06
Number of successes: 162
Number of failures: 0
Operation time to live: Forever

[SNIP

R4#


Path Enforcement

How does PfR modify Paths on the Border Routers:

PfR will create dynamic route-maps (dynamic PBR) on both Borders Routers (R4 and R5) to forward any flows matching the application:

  • over the PFR configured internal interfaces to the forwarding BR if it's the non-forwarding BR.
  • or over the external interfaces to the WAN if it is the forwarding BR.

In-order for the PBR based route control to be successful, the BRs have to be adjacent either through a direct connection or one hop away. This can be verified with the following command:


R3#sh pfr master
OER state: ENABLED and ACTIVE
  Conn Status: SUCCESS, PORT: 3949
  Version: 3.3
  Number of Border routers: 2
  Number of Exits: 2
  Number of monitored prefixes: 8 (max 5000)
  Max prefixes: total 5000 learn 2500
  Prefix count: total 8, learn 6, cfg 0
  PBR Requirements met                     <---- HERE
  Nbar Status: Inactive

Border           Status                UP/DOWN             AuthFail  Version  DOWN Reason
10.10.10.5       ACTIVE                UP       00:09:15          0  3.3
10.10.10.4       ACTIVE                UP       00:09:19          0  3.3

[SNIP]

What to check:

  • Look at PBR Requirement - should be met.


You can also check the Border Routers topology:


R3#sh pfr master border topology
Local Border   Local Interface   Remote Border  Remote Interface  Neighbor type
-------------------------------------------------------------------------------------------
10.10.10.5     Ethernet0/0       10.10.10.4     Ethernet0/0       Directly Connected
10.10.10.4     Ethernet0/0       10.10.10.5     Ethernet0/0       Directly Connected
PBR Requirements met
R3#


Dynamic route maps can be verified directly on the Border Routers:


R4#sh route-map dynamic detail
route-map OER_INTERNAL_RMAP, permit, sequence 0, identifier 2751463444
  Match clauses:
    ip address (access-lists): oer#1
      Extended IP access list oer#1
          268435455 permit ip any 10.1.10.0 0.0.0.255 dscp default (1570 matches)
          536870911 permit ip any 10.1.10.0 0.0.0.255 dscp af31 (2494 matches)
          1073741823 permit ip any 10.1.10.0 0.0.0.255 dscp ef (209 matches)
          2147483647 deny ip any any (8948 matches)
  Set clauses:
    ip next-hop 10.0.100.10
    interface Tunnel100
  Policy routing matches: 4273 packets, 3698452 bytes
route-map OER_INTERNAL_RMAP, permit, sequence 1, identifier 3456106517
  Match clauses:
    ip address (access-lists): oer#2
      Extended IP access list oer#2
          536870911 permit ip any 10.1.9.0 0.0.0.255 dscp ef (207 matches)
          1073741823 permit ip any 10.1.9.0 0.0.0.255 dscp af31 (2489 matches)
          2147483647 deny ip any any (6235 matches)
  Set clauses:
    ip next-hop 10.0.100.9
    interface Tunnel100
  Policy routing matches: 2696 packets, 2269648 bytes
route-map OER_INTERNAL_RMAP, permit, sequence 2, identifier 3791650838
  Match clauses:
    ip address (access-lists): oer#3
      Extended IP access list oer#3
          1073741823 permit ip any 10.1.9.0 0.0.0.255 dscp default
          2147483647 deny ip any any (1194 matches)
  Set clauses:
    ip next-hop 10.10.45.5
    interface Ethernet0/0
  Policy routing matches: 0 packets, 0 bytes
Current active dynamic routemaps = 1
R4#


What to check:

  • oer#1 route-map matches voice/video and critical apps to Branch R10. next-hop 10.0.100.10 is R10 over DMVPN1. This is expected as DMVPN1 is the preferred path for voice/video/critical
  • oer#1 route-map also matches best effort traffic going to Branch10. next-hop 10.0.100.10 is R10 over DMVPN1. This is a result of the load-balancing policy. Rest of best effort is on DMVPN2.
  • oer#2 route-map matches voice/video and critical apps to Branch R9. next-hop 10.0.100.9 is R9 over DMVPN1 (preferred path)


Conclusion

Performance Routing is an advanced technology that allows multiple deployments, one of them being the one described in this document. The use of an overlay topology (DMVPN) over any given transport allows flexibility while keeping the configuration exactly the same for PfR.

The configuration is straightforward and very efficient. Based on that, a customer can evolve to a more complex Performance Routing solution or can fine-tune the Traffic Class definition.



Rating: 4.9/5 (12 votes cast)

Personal tools