PfR:Solutions:NBAR

From DocWiki

Revision as of 10:50, 21 September 2012 by Jbarozet (Talk | contribs)
Jump to: navigation, search

Pfr-banner-solutions.png



PfR NBAR Based Application Control


Navigation


Contents



Introduction

The Performance Routing with NBAR/CCE Application Recognition feature introduces the ability to profile an application-based traffic class using Network-Based Application Recognition (NBAR). NBAR is a classification engine that recognizes and classifies a wide variety of protocols and applications, including web-based and other difficult-to-classify applications and protocols that use dynamic TCP/UDP port assignments. PfR uses NBAR to recognize and classify a protocol or application, and the resulting traffic classes are added to the PfR application database to be passively and/or actively monitored.

As a consequence PFR is capable of optimizing applications identified by Network-Based Application Recognition (NBAR). Using the “match traffic-class application nbar nbar-appl-name [nbar-appl-name ...] prefix-list prefix-listname” command or “match pfr learn list <list-name>” , PFR will be able to use NBAR to recognize applications and optimize them based on the policies installed on the Master Controller. Internally, PFR uses classmaps to control and optimize NBAR based applications. The list of supported NBAR identified applications is constantly being updated and an updated list should be obtained to determine its support.


PfR Network Topology Used

The central site has two Border Routers(R4 & R5), connected to two separate Service Providers.

The central site has two Border Routers, connected to two separate Service Providers using eBGP. R2, R4 and R5 are iBGP peers.

  • R3 is the Master Controller
  • R4 and R5 the Border Routers
  • Traffic Simulator tool is used between R1 and R11 to emulates traffic, both TCP and UDP.
  • R6 and R7 are delay generators that add delay/loss to the path through SP1 and SP2. By default, 100 ms through SP1 and 50 ms through SP2.
  • R1 and R11 are packet generators that send/receive traffic (http, ssh, etc).


Pfr-topology3.jpg


R2 is an iBGP peer for both border routers and as such has the BGP route table. PfR being not active, the parent routes are BGP based on R2, R4 and R5 and R5 is the preferred exit point (higher local preference applied on ingress on R5) as seen below:

Route to destination prefixes BRANCH1 (20.20.0.0/16):


R2#sh ip bgp 20.20.0.0/16
BGP routing table entry for 20.20.0.0/16, version 7
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Refresh Epoch 1
  200 20
    100.5.82.1 (metric 11) from 10.5.5.5 (10.5.5.5)
      Origin IGP, metric 0, localpref 200, valid, internal, best
R2#

Route to destination prefixes BRANCH2 (30.30.0.0/16):


R2#sh ip bgp 30.30.0.0/16
BGP routing table entry for 30.30.0.0/16, version 12
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Refresh Epoch 1
  200 30
    100.5.82.1 (metric 11) from 10.5.5.5 (10.5.5.5)
      Origin IGP, metric 0, localpref 200, valid, internal, best
R2#



Configuration

As usual, most of the configuration is done on the Master Controller.


Master Controller Configuration - R3

Basic configuration to establish session between MC and BR


! This is the basic configuration required to establish session between MC and BR.
! With this basic configuration, learning is enabled, route control is in place and
! load-balancing happens on all configured external interfaces.
!
! It includes
! – Key-chain configuration for authentication.
! – Specification of BR’s IP Address and internal/external interface on the BR.
!   If there is a tunnel between the BRs, it should be configured as internal
! - Specification of the Carrier name. Not used in the policies described hereafter.
! - Disabling auto-tunnels as BRs are L2 adjacent.
!
! Notes on the new defaults:
! - Automatic learning is enabled
! - mode route control is enabled
! - Maximum transmit utilization is 90%
! - Load-balancing is now used as the last resolver and cannot be disabled. Range is 20%.
!
!
key chain pfr
 key 0
  key-string cisco
!
pfr master
 logging
 !
 border 10.4.4.4 key-chain pfr
  interface Ethernet0/0 internal
  interface Ethernet0/1 external
   link-group SP1
 !
 border 10.5.5.5 key-chain pfr
  interface Ethernet0/0 internal
  interface Ethernet0/1 external
   link-group SP2
 !
 no mode auto-tunnels
!


Enable NetFlow version9 Export (optional)


! Flow Exporter Definition
! Where do I want my data sent?
!
flow exporter MYEXPORTER
 destination 10.151.1.131
 source Loopback0
 transport udp 9991
 option interface-table timeout 300
 option sampler-table timeout 300
 option application-table timeout 300
!
!
! Add NetFlow export to PfR
!
pfr master
 exporter MYEXPORTER
!


Enable logging


! Following configuration is to enable logging. 
! This will print PfR related syslog messages on the console.
!
pfr master
 logging
!


Define learning parameters, disable global learning


! By default now:
!
! - Automatic learning is enabled
!
! - Continuous learn cycle, each 1 minute duration 
!   periodic-interval = 0 (forever)
!   monitor-period = 1 (minutes)
!
! - traffic-class sorting based on ‘throughput’ at the end of
!   each learning cycle.
!
! - Anything traffic that doesn’t match the learn list will be
!   learned under global learn and will be optimized using default policy. 
!
! - In this example we decided to disable global learning 
!   configure a filter using a named access-list.
!   The goal here is to learn only branch traffic using learn list (described in the next section).
!
pfr master
 learn
  ! Disable Global learn
  !
  traffic-class filter access-list DENY_GLOBAL_LEARN_LIST
 ! 
!
! Access-list for disabling global learn.
!
ip access-list extended DENY_GLOBAL_LEARN_LIST
 deny ip any any
!


Define learn-list for an NBAR based application


! Following is a learn list configuration for NBAR based application control. 
! Learn-list is defined for a specific remote site BRANCH1.
! In the configuration below, http has been used as an example.
! You can tune the aggregation mask to match your need. 
! Here we used the default value of /24
! Sorting is based on throughput (default)
!
pfr master
 learn
 list seq 10 refname HTTP
   traffic-class application nbar http filter BRANCH1_PREFIX
   aggregation-type prefix-length 24
   throughput
!
!
! Define the prefix list of the branch
!
ip prefix-list BRANCH1_PREFIX seq 5 permit 20.20.0.0/16
!


Policy configuration for controlling NBAR applications and specific for a branch


! Following is a policy configuration to control NBAR applications 
! specific to one branch.
!
! This should be repeated for each branch. It includes
!
! – match command is to specify that this policy should be applied
!   to all the traffic-classes learned under list HTTP
!
! - Re-evaluate exit every 90 sec (periodic 90)
!
! – delay threshold is configured as 60 msec. The delay measured by PfR is
!   Round-Trip-Time.
!
! – Resolver setting is configured to set delay as the highest priority 
!
! - Load-balancing is now used as the last resolver (cannot be disabled for now)
!
! - monitor mode is set to fast. This means probe all external interfaces
!   all the time. When Out-of-Policy condition is detected on the current 
!   exit results on alternate exit is available for quick decision. In other 
!   modes alternate exits are probed only when current link is determined to 
!   be OOP. The fast mode helps in switching the path quickly when the 
!   problem is detected.
!
! – Probe frequency is set to 8 seconds and can be changed to a lesser value
!   if the application being controlled is critical.
!
!-  A forced echo probe has been setup
! 
! Default Values (new in 15.2(3)T and 3.6): 
! - holddown timer set to 90 to shorten the time needed to move to INPOLICY state
! 
!
pfr-map MYMAP 10
 match pfr learn list HTTP
 set periodic 90
 set mode select-exit good
 set delay threshold 60
 set mode monitor fast
 set resolve delay priority 1 variance 5
 set active-probe echo 20.9.9.9
 set probe frequency 8
!


Assign the policy to the PfR Master Controller


!
pfr master
 policy-rules MYMAP
!


Note: You can choose to replace the learnt list based NBAR application based learning with the configured application/prefix-list method as well. This would require the user to replace the match statement in the configured pfr-map:

From:


pfr-map MYMAP 10
 match pfr learn list HTTP
 (…)

To:


pfr-map MYMAP 10
 match traffic-class application nbar http prefix-list myprefix
 (…)
!
! where myprefix is a configured ip prefix-list:
!
ip prefix-list myprefix seq 5 permit 100.2.5.2/32


Border Routers Configuration – Routers R4 and R5


Following are common PfR configuration commands in the Border Router R4 and R5.


! This is the minimum configuration required on the BR’s. 
! It includes
! – Key-chain configuration for authentication.
! – Specification of MC’s IP Address and Local interface. The IP address
!   of the local interface will be used as source IP address in communicating
!   with MC.
!
key chain pfr
 key 0
  key-string cisco
!
pfr border
 local Ethernet0/0
 master 10.3.3.3 key-chain pfr
!



PfR Configuration Verification

Master Controller

The first step is to check the master controller configuration.


Verify the Border Routers, verify the parameters used (default and configured) and check the learn-list.


R3#sh pfr master
OER state: ENABLED and ACTIVE
  Conn Status: SUCCESS, PORT: 3949
  Version: 3.3
  Number of Border routers: 2
  Number of Exits: 2
  Number of monitored prefixes: 4 (max 5000)
  Max prefixes: total 5000 learn 2500
  Prefix count: total 4, learn 2, cfg 0
  PBR Requirements met
  Nbar Status: Active
  Auto Tunnel Mode: Off

Border           Status                UP/DOWN             AuthFail  Version  DOWN Reason
10.4.4.4         ACTIVE                UP       14:01:33          0  3.3
10.5.5.5         ACTIVE                UP       14:01:33          0  3.3

Global Settings:
  max-range-utilization percent 20 recv 0
  rsvp post-dial-delay 0 signaling-retries 1
  mode route metric bgp local-pref 5000
  mode route metric static tag 5000
  trace probe delay 1000
  logging 
  exit holddown time 60 secs, time remaining 0

Default Policy Settings:
  backoff 90 900 90
  delay relative 50
  holddown 90
  periodic 0
  probe frequency 56
  number of jitter probe packets 0
  mode route control 
  mode monitor both
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  trigger-log percentage 30

Learn Settings:
  current state : STARTED
  time remaining in current state : 80 seconds
  throughput
  no delay
  no inside bgp
  traffic-class filter access-list DENY_GLOBAL_LEARN_LIST
  monitor-period 1
  periodic-interval 0
  aggregation-type prefix-length 24
  prefixes 100 appls 100
  expire after time 720

  Learn-List seq 10 refname HTTP
    Configuration:
     Traffic-Class Application: http
     Filter: BRANCH1_PREFIX
     Aggregation-type: prefix-length 24
     Learn type: throughput
     Session count: 1000 Max count: 1000
     Policies assigned: 10
     Status: ACTIVE
    Stats:
     Traffic-Class Count: 2
R3#   

What to check:

  • Both Border Routers are up and running
  • Check to make sure that NBAR Status is Active.
  • Check to make sure that PBR Requirements are met
  • Number of automatically learned applications – 2 in this case
  • All default policy settings are displayed
  • Learn is started (current state : STARTED)
  • Learn-list status is ACTIVE and learn-type is throughput
  • Make sure that all the configured learn lists show up
  • For each learn-list, check the Traffic Class access-list (if configured), prefix-list (if configured) and the policy number associated (which is the pfr-map it refers to).
  • The Traffic Class count is also displayed which allows you to check whether the learning process worked well (here 2 applications is being learnt under HTTP learn list).


Check if the application is automatically learnt under each learn-list


R3#show pfr master learn list

 Learn-List seq 10 refname HTTP
   Configuration:
    Traffic-Class Application: http
    Filter: BRANCH1_PREFIX
    Aggregation-type: prefix-length 24
    Learn type: throughput
    Session count: 1000 Max count: 1000
    Policies assigned: 10
    Status: ACTIVE
   Stats:
    Traffic-Class Count: 2
    Traffic-Class Learned:
     Appl Prefix 20.20.1.0/24 http
     Appl Prefix 20.20.2.0/24 http
R3#


Check the status of NBAR application being controlled on the Master

This command is used to display information about the status of an application identified using NBAR for each PfR border router. The following partial output shows information about the status of applications identified using NBAR at PfR border routers R4 (10.4.4.4) and R5 (10.5.5.5). If the NBAR application is not supported on one or more border routers, then all the traffic classes related to that NBAR application are marked inactive and cannot be optimized using PfR.


R3#show pfr master nbar application
OER NBAR Applications Status:

NBAR Appl                  10.4.4.4        10.5.5.5
---------------------------------------------------
bgp                           Valid           Valid
bittorrent                    Valid           Valid
bridge                      Invalid         Invalid
bstun                       Invalid         Invalid
cdp                         Invalid         Invalid
citrix                        Valid           Valid
clns                        Invalid         Invalid
clns_es                     Invalid         Invalid
clns_is                     Invalid         Invalid
cmns                        Invalid         Invalid
compressedtcp               Invalid         Invalid
cuseeme                       Valid           Valid

[SNIP]

R3#



R3#show pfr master nbar application  | inc http
http                          Valid           Valid
secure-http                   Valid           Valid
R3#


Check the policy associated


R3#show pfr master policy 10
* Overrides Default Policy Setting
oer-map MYMAP 10
  sequence no. 8444249301975040, provider id 1, provider priority 30
    host priority 0, policy priority 10, Session id 0
  match oer learn list HTTP
  backoff 90 900 90
 *delay threshold 60
  holddown 90
 *periodic 90
 *probe frequency 8
  number of jitter probe packets 0
  mode route control 
 *mode monitor fast
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  next-hop not set
  forwarding interface not set
  trigger-log percentage 30
 *resolve delay priority 1 variance 5

  Forced Assigned Target List:
   active-probe echo 20.9.9.9 target-port 0
R3#  


Check the active probes


R3#show pfr master active-probes forced 
        OER Master Controller active-probes
Border   = Border Router running this Probe
Policy   = Forced target is configure under this policy
Type     = Probe Type
Target   = Target Address
TPort    = Target Port
N - Not applicable

The following Forced Probes are running:

Border          State    Policy             Type     Target          TPort Dscp 
10.4.4.4        ACTIVE   10                 echo     20.9.9.9            N defa
10.5.5.5        ACTIVE   10                 echo     20.9.9.9            N defa


R3#


Verify Traffic Class Statistics


As soon as you see:

R3#
*Sep 20 20:24:43.515: %OER_MC-5-NOTICE: Prefix Learning WRITING DATA
*Sep 20 20:24:43.589: %OER_MC-5-NOTICE: Prefix Learning STARTED
R3#


You should be able to see the traffic classes on the Master Controller. You will need a few cycles before having all applications in INPOLICY state.


Verify learnt Traffic Classes

On the Master Controller, you have all the Traffic Classes (learnt as well as statistics): Use show pfr master traffic-class to see the state of the application:

Traffic Classes are in HOLDDOWN state first:


R3#sh pfr master traffic-class 
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (packets-per-million), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------
20.20.1.0/24           http    N    N           N           N 0.0.0.0/0         
                          HOLDDOWN      @47          10.4.4.4 Et0/1           CCE     
               U        U        0        0        0        0       59        4
              51       51        0        0        N        N        N        N

20.20.2.0/24           http    N    N           N           N 0.0.0.0/0         
                          HOLDDOWN      @45          10.4.4.4 Et0/1           CCE     
               U        U        0        0        0        0       59        4
              51       51        0        0        N        N        N        N

R3#

Then moved into INPOLICY state:


R3#sh pfr master traffic-class 
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (packets-per-million), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------
20.20.1.0/24           http    N    N           N           N 0.0.0.0/0         
                          INPOLICY      @72          10.4.4.4 Et0/1           CCE     
               U        U        0        0        0        0       55        4
              49       50        0        0        N        N        N        N

20.20.2.0/24           http    N    N           N           N 0.0.0.0/0         
                          INPOLICY      @70          10.4.4.4 Et0/1           CCE     
               U        U        0        0        0        0       54        4
              49       50        0        0        N        N        N        N

R3# 

What to check:

  • Traffic Classes Appl_ID is http (NBAR)
  • state is INPOLICY (could be HOLDOWN and then INPOLICY after 90 sec)
  • Time column: remaining time in the current state
  • CurrBR: Current Border Router and its external interface used
  • 2 lines of results: 1st line is passive results and 2nd line is active results
  • Protocol: enforcement method used. Here CCE, means classmaps with a set next-hop (NBAR within PBR is not supported yet).


A closer look at the results

Let’s have a look at the show pfr master traffic-class output in more detail. We will use the show pfr master traffic-class performance command to do this. Note: This command is only available from 15.2(1)T


R3#show pfr master traffic-class performance ip any 20.20.1.0/24
==============================================================================================

Traffic-class:
 Destination Prefix : 20.20.1.0/24            Source Prefix    : 0.0.0.0/0
 Destination Port   : N/A                     Source Port      : N/A
 DSCP               : N                       Protocol         : N/A
 Application Name:  : http

 General:
   Control State                   : Controlled using CCE
   Traffic-class status            : INPOLICY
   Current Exit                    : BR 10.4.4.4 interface Et0/1, Tie breaker was Delay
   Time on current exit            : 0d 0:2:2
   Time remaining in current state : @52 seconds
   Traffic-class type              : Learned
   Improper config                 : None

 Last Out of Policy event:
   Exit                            : BR 10.4.4.4 interface Et0/1
   Reason                          : Delay
   Time since Out of Policy event  : 0d 0:2:2
   Active Delay Performance        : 49 msecs
   Active Delay Threshold          : 60 msecs

 Average Passive Performance Current Exit: (Average for last 5 minutes)
   Unreachable            : 0% -- Threshold: 50%
   Delay                  : 0 msecs -- Threshold: 60 msecs
   Loss                   : 0% -- Threshold: 10%
   Egress BW              : 55 kbps
   Ingress BW             : 4 kbps
   Time since last update : 0d 0:1:35

 Average Active Performance Current Exit: (Average for last 5 minutes)
   Unreachable            : 0% -- Threshold: 50%
   Delay                  : 49 msec -- Threshold: 60 msec

 Last Resolver Decision:
   BR              Interface    Status       Reason       Performance Threshold
   --------------- ------------ ------------ ------------ ----------- ---------
   10.5.5.5        Et0/1        Eliminated   Delay        105 msecs    60 msecs    
   10.4.4.4        Et0/1        Best Exit    Delay        51 msecs     60 msecs    

==============================================================================================
R3#

The sections under:

  • Traffic-class: -- displays the application-prefix that is being learnt or monitored.
  • General: -- will display whether the application is being controlled, the controlling protocol, and the state. In the above output the application is INPOLICY and is being controlled using CCE, which is the class-map based control that is used along with a set next hop, to enforce the path.
  • The Active and Passive Performance measurements are also displayed in the sections that follow.
  • Lastly, you will see the Resolver decision that was used to select the best exit. In the example above, we have a delay threshold of 60 ms configured in the policy MYMAP. The delay being calculated on BR(R5)- 10.5.5.5, exit Et0/1 is 105 msec and therefore it is being eliminated. The delay calculated on BR(R4)- 10.4.4.4, exit Et0/1 is 51 msec, which is within the configured delay threshold and therefore it is chosen as the best exit.



Border Routers

Check the netflow cache on the Border Routers

On Border Router that is supposed to learn the flow, issue “show ip cache flow” and verify that the flow is being learnt.

Note: There is no need to explicitly configured NetFlow, the Master Controller tells the BRs to enable NetFlow.


How does PfR modify Paths on the Border Routers

PfR will create classmaps (CCE) on the non-forwarding Borders to forward any flows matching the application over the PFR configured internal interfaces to the forwarding Border. Finally on the forwarding Border, there will be another classmap directing traffic for that application out of the border’s external exit, that is chosen as the best exit.

In-order for the CCE based route control to be successful, the BRs have to be adjacent either through a direct connection or one hop away.

The primary next-hop from BGP is R5. Without PfR, traffic is flowing through R5 and SP2. Let's check on the Border Router - R5, where we clearly see that packets are being sent out to R4.


R5#show pfr border routes cce
Class-map oer-class-acl-oer_cce#2-stile-http, permit, sequence 0, mask 24
  Match clauses:
    ip address (access-list): oer_cce#2
    stile: http
  Set clauses:
    ip next-hop 10.4.5.4
    interface Ethernet0/0
  Statistic:
    Packet-matched: 18491

R5#

R5#show ip access-lists dynamic oer_cce#2
Extended IP access list oer_cce#2
    10 permit ip any 20.20.2.0 0.0.0.255 (9579 matches)
    20 permit ip any 20.20.1.0 0.0.0.255 (9801 matches)
R5#


The packets are being finally sent out of R4’s external exit since that was chosen as the best exit.


R4#show pfr border routes cce
Class-map oer-class-acl-oer_cce#2-stile-http, permit, sequence 0, mask 24
  Match clauses:
    ip address (access-list): oer_cce#2
    stile: http
  Set clauses:
    ip next-hop 100.4.81.1
    interface Ethernet0/1
  Statistic:
    Packet-matched: 76

R4#

R4#show ip access-lists dynamic oer_cce#2
Extended IP access list oer_cce#2
    10 permit ip any 20.20.2.0 0.0.0.255 (54 matches)
    20 permit ip any 20.20.1.0 0.0.0.255 (22 matches)
R4#



Further Debugging Tips

The following set of debugs might help narrow down any issues seen while troubleshooting NBAR CCE control within PFR:

  • On the Border

debug pfr border cce [detail]
debug pfr border nbar [detail]


  • On the Master Controller

debug pfr master prefix [appl] [detail]


  • Further, if you have PFR Master logging configured, you should be able to see Notice messages for indications of other actions being taken by PFR.
  • The following debugs can be turned on the Master Controller to see if netflow or active probing updates are being received from the borders for ingress/egress flows.

MC#debug pfr master collector ?
  active-probes  Display PfR active probe debugs
  netflow            Display PfR NetFlow debugs


Conclusion

The configuration examples provided above explain how applications can be learnt and optimized using learn lists. The same configuration can be extended to even consider configured NBAR based application control – with prefix-lists associated with them. The number of applications being optimized can be fine tuned based on the specific requirement.


Rating: 5.0/5 (2 votes cast)

Personal tools