Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Unicast Traffic

From DocWiki

(Difference between revisions)
Jump to: navigation, search
(Removing all content from page)
Line 1: Line 1:
-
{{Template:Required Metadata}}
 
-
This article provides information on how to troubleshoot unicast packet flow traffic issues for the M1 Series modules.
 
-
{| align="right" border="1"
 
-
|align="center"|'''Guide Contents'''
 
-
|-
 
-
|[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Overview|Troubleshooting Overview]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Installs, Upgrades, and Reboots|Troubleshooting Installs, Upgrades, and Reboots]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Licensing|Troubleshooting Licensing]]'<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VDCs|Troubleshooting VDCs]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting CFS|Troubleshooting CFS]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Ports|Troubleshooting Ports]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs|Troubleshooting vPCs]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VLANs|Troubleshooting VLANs]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting STP|Troubleshooting STP]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Routing|Troubleshooting Routing]]<br>''Troubleshooting Unicast Traffic(this section)''<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting WCCP|Troubleshooting WCCP]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Memory|Troubleshooting Memory]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Packet Flow Issues|Troubleshooting Packet Flow Issues]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Before Contacting Technical Support|Before Contacting Technical Support]]<br>[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Tools and Methodology|Troubleshooting Tools and Methodology]]
 
-
|}
 
-
 
-
__TOC__
 
-
 
-
==Step 1: Packet is Received into Interface from Wire==
 
-
 
-
During this step, the packet is received into the Nexus 7000 port. When troubleshooting this step, we want to look to ensure there is transceiver interoperability, and validate whether we are seeing any errors on the interface. We do this via using the following commands
 
-
 
-
* '''show interface''' ''interface''
 
-
* '''show interface''' ''interface'' '''transceiver'''
 
-
 
-
PHX2-N7K-1# '''show interface e1/1'''
 
-
  '''Ethernet1/1 is up'''
 
-
  Hardware: 10000 Ethernet, address: 0024.986c.00b0 (bia 0024.986c.00b0)
 
-
  Description: N7K-vdc-1 connecting to core 6506
 
-
  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
 
-
    reliability 255/255, txload 1/255, rxload 1/255
 
-
  Encapsulation ARPA
 
-
  Port mode is trunk
 
-
  full-duplex, 10 Gb/s, media type is 10g
 
-
  Beacon is turned off
 
-
  Auto-Negotiation is turned off
 
-
  Input flow-control is off, output flow-control is off
 
-
  '''Rate mode is shared'''
 
-
  Switchport monitor is off
 
-
  Last link flapped 7week(s) 4day(s)
 
-
  Last clearing of "show interface" counters never
 
-
  '''1 minute input rate''' 13056 bits/sec, 9 packets/sec
 
-
  '''1 minute output rate''' 4608 bits/sec, 0 packets/sec
 
-
  Rx
 
-
    341190251 input packets 276211313 unicast packets 52112947 multicast packets
 
-
    12865991 broadcast packets 0 jumbo packets 0 storm suppression packets
 
-
    94295027129 bytes
 
-
  Tx
 
-
    462437316 output packets 85121 multicast packets
 
-
    188251 broadcast packets 0 jumbo packets
 
-
    648159081064 bytes
 
-
    '''0 input error 0 short frame 0 watchdog
 
-
    0 no buffer 0 runt 0 CRC 0 ecc
 
-
    0 overrun  0 underrun 0 ignored 0 bad etype drop
 
-
    0 bad proto drop 0 if down drop 0 input with dribble
 
-
    0 input discard
 
-
    0 output error 0 collision 0 deferred
 
-
    0 late collision 0 lost carrier 0 no carrier
 
-
    0 babble
 
-
    0 Rx pause 0 Tx pause'''
 
-
  1 interface resets
 
-
 
-
PHX2-N7K-1# '''show interface e1/1 transceiver details '''
 
-
  Ethernet1/1
 
-
    sfp is present
 
-
    name is CISCO-AVAGO    '''<<< If this says type is (unknown), it is not supported.'''
 
-
    part number is SFBR-7700SDZ   
 
-
    revision is B4 
 
-
    serial number is AGD12434116   
 
-
    nominal bitrate is 10300 MBits/sec
 
-
    Link length supported for 50/125um fiber is 82 m(s)
 
-
    Link length supported for 62.5/125um fiber is 26 m(s)
 
-
    cisco id is --
 
-
    cisco extended id number is 4
 
-
 
-
          SFP Detail Diagnostics Information (internal calibration)
 
-
  ----------------------------------------------------------------------------
 
-
                                    Alarms                  Warnings
 
-
                                High        Low        High          Low
 
-
  ----------------------------------------------------------------------------
 
-
  Temperature  45.46 C        75.00 C    -5.00 C    70.00 C        0.00 C
 
-
  Voltage        3.28 V        3.63 V      2.97 V      3.46 V        3.13 V
 
-
  Current        6.92 mA      10.50 mA    2.50 mA    10.50 mA      2.50 mA
 
-
  Tx Power      -2.75 dBm      1.69 dBm  -11.30 dBm  -1.30 dBm    -7.30 dBm
 
-
  Rx Power      -2.43 dBm      1.99 dBm  -13.97 dBm  -1.00 dBm    -9.91 dBm
 
-
  Transmit Fault Count = 0
 
-
  ----------------------------------------------------------------------------
 
-
  Note: ++  high-alarm; +  high-warning; --  low-alarm; -  low-warning
 
-
 
-
==Step 2: Linksec Decryption Occurs, 1st stage Port QoS==
 
-
 
-
In step 2, Linksec decryption occurs as well as receive side stage 1 QoS.
 
-
 
-
It is important to step back and evaluate the difference between stage 1 and stage 2 QoS. The difference is that some ports can be configured in shared mode, whereas some can be configured in dedicated mode, on the 10G modules. What this means, is that there is 10g of bandwidth that can be dedicated to a port or shared amongst ports (4 ports, on the m132 module).
 
-
 
-
When running in shared mode, there exists a chance for contention accessing the 10g bandwidth through the 4:1 Mux. To alleviate this, some QoS intelligence was passed down to the 4:1 Mux which aggregates the ports.
 
-
 
-
In dedicated mode, there is no QoS applied at the Mux, instead, all traffic is processed in phase 2 QoS. To summarize, in shared mode, 1st stage QoS ensures fair access to the 10g of port bandwidth. In both shared and dedicated mode, 2nd stage QoS occurs to provide ingress queuing to the system.
 
-
 
-
For the ingress QoS, we are concerned about the Receive side QoS parameters in the show queuing command.
 
-
 
-
Use  the '''show policy-map''' command to see per queue dropped packets.
 
-
 
-
The commands to troubleshoot Linksec and Port QoS are as follows:
 
-
 
-
* '''show cts interface''' [''all'' | ''interface'']
 
-
* '''show queuing interface''' ''interface''
 
-
* '''show policy-map interface''' (for per queue drop)
 
-
 
-
 
-
switch# '''show cts interface all Working Example'''
 
-
CTS Information for Interface Ethernet2/24:
 
-
    CTS is enabled, mode:  CTS_MODE_DOT1X
 
-
    IFC state:              CTS_IFC_ST_CTS_OPEN_STATE
 
-
    Authentication Status:  CTS_AUTHC_SUCCESS
 
-
      Peer Identity:        india1
 
-
      Peer is:              CTS Capable
 
-
      802.1X role:          CTS_ROLE_AUTH
 
-
      Last Re-Authentication:
 
-
    Authorization Status:  CTS_AUTHZ_SUCCESS
 
-
      PEER SGT:            2
 
-
      Peer SGT assignment:  Trusted
 
-
      Global policy fallback access list:
 
-
    SAP Status:            '''CTS_SAP_SUCCESS'''
 
-
      Configured pairwise ciphers: GCM_ENCRYPT
 
-
      Replay protection: Enabled
 
-
      Replay protection mode: Strict
 
-
      Selected cipher: GCM_ENCRYPT
 
-
      Current receive SPI: '''sci:1b54c1fbff0000 an:0'''
 
-
      Current transmit SPI: '''sci:1b54c1fc000000 an:0'''
 
-
 
-
 
-
PHX2-N7K-1# '''show cts interface eth 1/8 Broken Example'''
 
-
CTS Information for Interface Ethernet1/8:
 
-
    CTS is enabled, mode:  CTS_MODE_MANUAL
 
-
    IFC state:              Unknown
 
-
    Authentication Status:  CTS_AUTHC_INIT
 
-
      Peer Identity:       
 
-
      Peer is:              '''Not CTS Capable'''
 
-
      802.1X role:          CTS_ROLE_UNKNOWN
 
-
      Last Re-Authentication:
 
-
    Authorization Status:  CTS_AUTHZ_INIT
 
-
      PEER SGT:            0
 
-
      Peer SGT assignment:  Not Trusted
 
-
    SAP Status:            CTS_SAP_INIT
 
-
      Configured pairwise ciphers:
 
-
      Replay protection:
 
-
      Replay protection mode:
 
-
      Selected cipher:
 
-
      Current receive SPI:
 
-
      Current transmit SPI:
 
-
 
-
 
-
 
-
PHX2-N7K-1# '''show queuing int eth 1/1'''
 
-
  Interface Ethernet1/1 TX Queuing strategy: Weighted Round-Robin
 
-
  Port QoS is enabled
 
-
    Queuing Mode in TX direction: mode-cos
 
-
    Transmit queues [type = 1p7q4t]
 
-
    Queue Id                      Scheduling  Num of thresholds
 
-
    _____________________________________________________________
 
-
      1p7q4t-out-q-default        WRR            04
 
-
      1p7q4t-out-q2              WRR            04
 
-
      1p7q4t-out-q3              WRR            04
 
-
      1p7q4t-out-q4              WRR            04
 
-
      1p7q4t-out-q5              WRR            04
 
-
      1p7q4t-out-q6              WRR            04
 
-
      1p7q4t-out-q7              WRR            04
 
-
      1p7q4t-out-pq1              Priority        04
 
-
 
-
  Configured WRR
 
-
      WRR bandwidth ratios:  25[1p7q4t-out-q-default] 15[1p7q4t-out-q2] 12[1p7q4t-out-q3]
 
-
  12[1p7q4t-out-q4] 12[1p7q4t-out-q5] 12[1p7q4t-out-q6] 12[1p7q4t-out-q7]
 
-
  WRR configuration read from HW
 
-
      WRR bandwidth ratios:  25[1p7q4t-out-q-default] 15[1p7q4t-out-q2] 11[1p7q4t-out-q3]
 
-
  11[1p7q4t-out-q4] 11[1p7q4t-out-q5] 11[1p7q4t-out-q6] 11[1p7q4t-out-q7]
 
-
 
-
  Configured queue-limit ratios
 
-
      queue-limit ratios:    78[1p7q4t-out-q-default] 1[1p7q4t-out-q2] 1[1p7q4t-out-q3]
 
-
  *1[1p7q4t-out-q4] *1[1p7q4t-out-q5] *1[1p7q4t-out-q6] *1[1p7q4t-out-q7] 16[1p7q4t-out-pq1]
 
-
  * means unused queue with mandatory minimum queue-limit
 
-
 
-
  queue-limit ratios configuration read from HW
 
-
      queue-limit ratios:    78[1p7q4t-out-q-default] 1[1p7q4t-out-q2] 1[1p7q4t-out-q3]
 
-
*1[1p7q4t-out-q4] *1[1p7q4t-out-q5] *1[1p7q4t-out-q6] *1[1p7q4t-out-q7] 16[1p7q4t-out-pq1]
 
-
  * means unused queue with mandatory minimum queue-limit
 
-
 
-
  Thresholds:
 
-
      COS    Queue                      Threshold Type      Min    Max
 
-
      __________________________________________________________________
 
-
      0    1p7q4t-out-q-default            DT              100    100
 
-
      1    1p7q4t-out-q-default            DT              100    100
 
-
      2    1p7q4t-out-q-default            DT              100    100
 
-
      3    1p7q4t-out-q-default            DT              100    100
 
-
      4    1p7q4t-out-q-default            DT              100    100
 
-
      5    1p7q4t-out-pq1                  DT              100    100
 
-
      6    1p7q4t-out-pq1                  DT              100    100
 
-
      7    1p7q4t-out-pq1                  DT              100    100
 
-
 
 
-
  '''Interface Ethernet1/1 RX Queuing strategy: Weighted Round-Robin'''
 
-
    '''Queuing Mode in RX direction: mode-cos'''
 
-
    '''Receive queues [type = 8q2t]'''
 
-
  ''' Port Cos not configured'''
 
-
    Queue Id                      Scheduling  Num of thresholds
 
-
    ____________________________________________________________
 
-
      8q2t-in-q-default          WRR            02
 
-
      8q2t-in-q2                  WRR            02
 
-
      8q2t-in-q3                  WRR            02
 
-
      8q2t-in-q4                  WRR            02
 
-
      8q2t-in-q5                  WRR            02
 
-
      8q2t-in-q6                  WRR            02
 
-
      8q2t-in-q7                  WRR            02
 
-
      8q2t-in-q1                  WRR            02
 
-
 
-
  Configured WRR
 
-
      WRR bandwidth ratios:  20[8q2t-in-q-default] 0[8q2t-in-q2] 0[8q2t-in-q3]
 
-
0[8q2tin-q4] 0[8q2t-in-q5] 0[8q2t-in-q6] 0[8q2t-in-q7] 80[8q2t-in-q1]
 
-
  WRR configuration read from HW
 
-
      WRR bandwidth ratios:  20[8q2t-in-q-default] 0[8q2t-in-q2] 0[8q2t-in-q3]
 
-
0[8q2t-in-q4] 0[8q2t-in-q5] 0[8q2t-in-q6] 0[8q2t-in-q7] 80[8q2t-in-q1]
 
-
 
-
  No queue-limit ratios user configuration
 
-
  ________________________________________
 
-
 
-
  queue-limit ratios configuration read from HW
 
-
      queue-limit ratios:    100[8q2t-in-q-default] 100[8q2t-in-q2] 100[8q2t-in-q3]
 
-
100[8q2t-in-q4] 100[8q2t-in-q5] 100[8q2t-in-q6] 100[8q2t-in-q7] 100[8q2t-in-q1]
 
-
 
-
  Thresholds:
 
-
      COS    Queue                      Threshold Type      Min    Max
 
-
      __________________________________________________________________
 
-
      0    8q2t-in-q-default              DT              100    100
 
-
      1    8q2t-in-q-default              DT              100    100
 
-
      2    8q2t-in-q-default              DT              100    100
 
-
      3    8q2t-in-q-default              DT              100    100
 
-
      4    8q2t-in-q-default              DT              100    100
 
-
      5    8q2t-in-q1                      DT              100    100
 
-
      6    8q2t-in-q1                      DT              100    100
 
-
      7    8q2t-in-q1                      DT              100    100
 
-
 
-
 
-
 
-
PHX2-N7K-1# '''show policy-map interface eth 1/2'''
 
-
Global statistics status :  enabled
 
-
Ethernet1/2
 
-
  Service-policy (queuing) '''input:'''  default-in-policy
 
-
    policy statistics status:  enabled
 
-
 
-
    Class-map (queuing):  in-q1 (match-any)
 
-
      queue-limit percent 50
 
-
      bandwidth percent 80
 
-
      '''queue dropped pkts : 0 '''
 
-
 
-
    Class-map (queuing):  in-q-default (match-any)
 
-
      queue-limit percent 50
 
-
      bandwidth percent 20
 
-
      '''queue dropped pkts : 0'''
 
-
 
-
  Service-policy (queuing) '''output: '''  default-out-policy
 
-
    policy statistics status:  enabled
 
-
 
-
    Class-map (queuing):  out-pq1 (match-any)
 
-
      priority level 1
 
-
      queue-limit percent 16
 
-
      '''queue dropped pkts : 0'''
 
-
 
-
    Class-map (queuing):  out-q2 (match-any)
 
-
      queue-limit percent 1
 
-
    ''' queue dropped pkts : 0'''
 
-
 
-
    Class-map (queuing):  out-q3 (match-any)
 
-
      queue-limit percent 1
 
-
    '''  queue dropped pkts : 0 '''
 
-
 
-
    Class-map (queuing):  out-q-default (match-any)
 
-
      queue-limit percent 82
 
-
      bandwidth remaining percent 25
 
-
    ''' queue dropped pkts : 0'''
 
-
 
-
 
-
==Step 3: Second Stage Port QoS Occurs==
 
-
 
-
For the ingress QoS, we are concerned about the Receive side QoS parameters in the show queuing command.
 
-
 
-
Use the '''show policy-map''' command to view queue drops .
 
-
 
-
The commands to troubleshoot Port QoS are
 
-
 
-
* '''show queuing interface''' ''interface''
 
-
* '''show policy-map interface'''
 
-
 
-
 
-
==Step 4:  Layer 2 Source/Destination MAC Processing==
 
-
 
-
In this step, the ASIC submits the packet headers to theLayer 2 engine for lookup, and the  Layer 2 engine performs source/destination MAC processing.
 
-
 
-
To validate forwarding of the Layer 2 engine, we should first look at the centralized mac table aggregated on the supervisor to validate whether the mac addresses are correlated as we expect them, and assigned to the ports where we expect the Mac’s to reside.
 
-
 
-
Based off of this, we can then validate the hardware programming on the ingress linecard to validate that our mac address table is properly programmed into the hardware based Layer 2 engine on the linecard.
 
-
 
-
We first will look at the mac address table, then we can ensure programming is properly occurring in the hardware table.
 
-
 
-
The commands used to accomplish this are as follows:
 
-
* '''show mac address-table'''
 
-
* '''show hardware mac address-table''' ''module'' '''interface''' ''interface''
 
-
 
-
To drill down on a specific MAC address, we can use the grep function with these commands to validate the mac is associated with a particular port, and that the hardware programming reflects that.
 
-
 
-
* '''show mac address-table | grep''' ''macaddress''
 
-
* '''show hardware mac address-table''' ''module'' '''interface''' ''interface'' | '''grep''' ''macaddress''
 
-
 
-
{note|When evaluating the Hardware mac table, if the Index is set to 0x00400, or the GM bit is set to “1”, that traffic will be routed. For example, you will see the index set to 0x00400 and GM bit set to 1 for traffic destined to the mac address local to the device}
 
-
 
-
PHX2-N7K-1# '''show mac address-table'''
 
-
Legend:
 
-
        * - primary entry, G - Gateway MAC, (R) - Routed MAC
 
-
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
 
-
    VLAN    MAC Address      Type      age    Secure  NTFY  Ports
 
-
---------+-----------------+--------+---------+------+------+----------------
 
-
G    -    0023.ac67.dd41    static      -    False  False sup-eth1(R)
 
-
G 5        0023.ac67.dd41    static      -    False  False sup-eth1(R)
 
-
* 5        0000.0c07.ac01    dynamic  0        False  False Eth1/1
 
-
* 5        000c.2943.a67e    dynamic  180      False  False Eth1/1
 
-
* 5        '''000c.294b.c5ca'''    dynamic  0        False  False '''Eth1/1'''
 
-
* 5        000d.ece2.0640    dynamic  180      False  False Eth1/1
 
-
* 5        0013.5f32.aa80    dynamic  0        False  False Eth1/1
 
-
* 5        0018.8b45.41b7    dynamic  0        False  False Eth1/1
 
-
* 5        0019.bb2f.4871    dynamic  0        False  False Eth1/1
 
-
* 5        0019.bbe5.f3b8    dynamic  1230    False  False Eth1/1
 
-
* 5        001a.4b33.ccdc    dynamic  1080    False  False Eth1/1
 
-
* 5        001a.4ba8.6a9c    dynamic  1680    False  False Eth1/1
 
-
* 5        001b.210a.87f9    dynamic  600      False  False Eth1/1
 
-
* 5        001b.d46f.70e0    dynamic  60      False  False Eth1/1
 
-
* 5        001c.c4e5.ac9a    dynamic  150      False  False Eth1/1
 
-
* 5        0023.ac64.6f7c    dynamic  1230    False  False Eth1/1
 
-
* 5        0024.986d.21c8    dynamic  270      False  False Eth1/1
 
-
 
-
 
-
PHX2-N7K-1# sho hardware mac address-table 1 int eth1/1
 
-
Valid| PI |  BD  |      MAC      |  Index | Stat| SW | Modi| Age | Tmr | GM | Sec|
 
-
  TR | NT | RM | RMA | Cap | Fld | Always    |    |      |              |
 
-
      |  ic |    | fied| Byte| Sel |    | ure| AP | FY |    |    | TURE|    |  Learn
 
-
-----+----+-------+---------------+--------+-----+----+-----+-----+-----+----+--
 
-
--+----+----+----+-----+-----+-----+--------
 
-
  1    1    2    '''000c.294b.c5ca'''  0x00422    0    3    0    67    1    0   
 
-
0    0    0    0    0    0    0  0
 
-
  1    1    2    0050.567e.58e6  0x00422    0    3    0    68    1    0   
 
-
0    0    0    0    0    0    0  0
 
-
  1    1    2    0050.56aa.6067  0x00422    0    3    0    67    1    0   
 
-
0    0    0    0    0    0    0  0
 
-
  1    1    2    00c0.b72e.cfa0  0x00422    0    3    0    67    1    0   
 
-
0    0    0    0    0    0    0  0
 
-
  1    1    2    0018.8b45.41b7  0x00422    0    3    0    68    1    0   
 
-
0    0    0    0    0    0    0  0
 
-
  1    1    2    0013.5f32.aa80  0x00422    0    3    0    68    1    0   
 
-
0    0    0    0    0    0    0  0
 
-
  1    1    2    0050.56aa.75ca  0x00422    0    3    0    64    1    0   
 
-
0    0    0    0    0    0    0  0
 
-
  1    1    2    00a0.9811.a233  0x00422    0    3    0    39    1    0   
 
-
0    0    0    0    0    0    0  0
 
-
 
-
 
-
PHX2-N7K-1# '''show mac address-table | grep 000c.294b.c5ca'''   
 
-
* 5        000c.294b.c5ca    dynamic  150      False  False '''Eth1/1'''
 
-
 
-
 
-
PHX2-N7K-1# '''show hardware mac address-table 1 int eth 1/1 | grep 000c.294b.c5ca'''
 
-
 
 
-
  1    1    2    '''000c.294b.c5ca  0x00422'''    0    3    0    67    1    0   
 
-
0    0    0    0    0    0    0  0
 

Revision as of 15:13, 7 October 2010

Rating: 4.0/5 (10 votes cast)

Personal tools