DocWiki - User contributions [en]

Cisco Nexus 7000 NX-OS/IOS Comparison Tech Notes

2012-03-06T18:45:37Z

Sukumars:

{{Template:Required Metadata}}
Cisco Nexus switches use the Cisco NX-OS Software. These comparisons are based on Cisco Nexus 7000 switches.

The following article compares default configuration differences between Cisco NX-OS Software and Cisco IOS Software.

* [[Cisco NX-OS/IOS Software Default Configuration Differences]]
* [http://docwiki.cisco.com/wiki/Catalyst_to_Nexus_Migration Catalyst 6500 IOS to NxOS Migration tool]

The following articles compare Cisco NX-OS Software with Cisco IOS Software for common features:

* [[Cisco NX-OS/IOS Configuration Fundamentals Comparison]]
* [[Cisco NX-OS/IOS BGP (Basic) Comparison]]
* [[Cisco NX-OS/IOS BGP (Advanced) Comparison]]
* [[Cisco NX-OS/IOS DHCP Relay Comparison]]
* [[Cisco NX-OS/IOS EIGRP Comparison]]
* [[Cisco NX-OS/IOS HSRP Comparison]]
* [[Cisco NX-OS/IOS Interface Comparison]]
* [[Cisco NX-OS/IOS Layer-3 Virtualization Comparison]]
* [[Cisco NX-OS/IOS Multicast Comparison]]
* [[Cisco NX-OS/IOS Netflow Comparison]]
* [[Cisco NX-OS/IOS OSPF Comparison]]
* [[Cisco NX-OS/IOS Port-Channel Comparison]]
* [[Cisco NX-OS/IOS SPAN Comparison]]
* [[Cisco NX-OS/IOS STP Comparison]]
* [[Cisco NX-OS/IOS TACACS+, RADIUS, and AAA Comparison]]

[[Category:Nexus 7000 Series Switches]]

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide

2011-11-30T21:36:49Z

Sukumars:

{{Template:Required Metadata}}
This article provides guidance for troubleshooting issues that may appear when using Cisco Nexus 7000 Series. This article introduces tools and methodologies to recognize a problem, determine its cause, and find possible solutions. However, these documentation helps only in basic troubleshooting.

'''We encourage users to review the Cisco Live Presentation for detailed troubleshooting for Nexus 7000.'''

Sections of this presentation covers, both platform independent, and platform specific step by step
troubleshooting for most common issues. Access to this presentation is available FREE. Follow the below instructions to access the
presentation.

1. Visit https://www.ciscolivevirtual.com/

2. Register for free.

3. Click on "Cisco Live Virtual" link.

4. Click on the ‘Sessions’ Tab on top, and select ‘2011 Sessions Catalog’

5. In the search box, type “BRKCRS-3144” and Submit search.

6. Select the session. You can either View the Session (or) download the pdf

{| align="center" border="1"
|align="left"|Welcome to '''Cisco DocWiki'''. We encourage [http://tools.cisco.com/RPF/register/register.do registered Cisco.com users] to contribute to this wiki to improve Cisco product documentation. Note that you cannot log in to DocWiki with Cisco.com "guest" account credentials. 
See [[DocWiki:Terms_of_use|Terms of Use]] and [[DocWiki:About|About DocWiki]] for more information about Cisco DocWiki. 
Select the "edit" tab to edit an article or select the "discussion" tab to submit questions or comments about the article. 

Click [http://www.cisco.com/en/US/products/ps9402/prod_troubleshooting_guides_list.html here] to return to the Cisco Nexus 7000 Series documentation on www.cisco.com.
|}

__TOC__

==Audience and Generating a PDF of This Guide==

This article is for experienced network administrators who configure and maintain NX-OS devices.

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Book PDF]]

==Organization==

This article is organized into the following sections:

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Overview|Troubleshooting Overview]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Installs, Upgrades, and Reboots|Troubleshooting Installs, Upgrades, and Reboots]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Licensing|Troubleshooting Licensing]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VDCs|Troubleshooting VDCs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting CFS|Troubleshooting CFS]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Ports|Troubleshooting Ports]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs|Troubleshooting vPCs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VLANs|Troubleshooting VLANs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting STP|Troubleshooting STP]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Routing|Troubleshooting Routing]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Unicast Traffic|Troubleshooting Unicast Traffic]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Multicast Traffic|Troubleshooting Multicast Traffic]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting WCCP|Troubleshooting WCCP]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Memory|Troubleshooting Memory]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Packet Flow Issues|Troubleshooting Packet Flow Issues]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Before Contacting Technical Support|Before Contacting Technical Support]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Tools and Methodology|Troubleshooting Tools and Methodology]]

[[Category:Nexus 7000 Series Switches]]

[[Category:Troubleshooting]]

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Overview

2011-11-30T21:35:56Z

Sukumars:

{{Template:Required Metadata}}
This article introduces the basic concepts, methodology, and general troubleshooting guidelines for problems that may occur when configuring and using Cisco NX-OS.

'''We encourage users to review the Cisco Live Presentation for detailed troubleshooting for Nexus 7000.'''

Sections of this presentation covers, both platform independent, and platform specific step by step troubleshooting for most common issues. Access to this presentation is available FREE. Follow the below instructions to access the presentation.

1. Visit https://www.ciscolivevirtual.com/

2. Register for free.

3. Click on "Cisco Live Virtual" link.

4. Click on the ‘Sessions’ Tab on top, and select ‘2011 Sessions Catalog’

5. In the search box, type “BRKCRS-3144” and Submit search.

6. Select the session. You can either View the Session (or) download the pdf
'''

{| align="right" border="1"
|align="center"|'''Guide Contents'''
|-
|''Troubleshooting Overview (this section)'' [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Installs, Upgrades, and Reboots|Troubleshooting Installs, Upgrades, and Reboots]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Licensing|Troubleshooting Licensing]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VDCs|Troubleshooting VDCs]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting CFS|Troubleshooting CFS]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Ports|Troubleshooting Ports]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs|Troubleshooting vPCs]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VLANs|Troubleshooting VLANs]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting STP|Troubleshooting STP]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Routing|Troubleshooting Routing]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Unicast Traffic|Troubleshooting Unicast Traffic]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting WCCP|Troubleshooting WCCP]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Memory|Troubleshooting Memory]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Packet Flow Issues|Troubleshooting Packet Flow Issues]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Before Contacting Technical Support|Before Contacting Technical Support]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Tools and Methodology|Troubleshooting Tools and Methodology]]
|}

__TOC__

== Overview of the Troubleshooting Process ==

To troubleshoot your network, follow these general steps:
# Maintain a consistent Cisco NX-OS release across all your devices.
# See the [http://www.cisco.com/en/US/products/ps9372/prod_release_notes_list.html Cisco NX-OS release notes] for your Cisco NX-OS release for the latest features, limitations, and caveats.
# Enable system message logging. See [[#System Messages|System Messages]].
# Troubleshoot any new configuration changes after implementing the change.
# Gather information that defines the specific symptoms. See [[#Gathering Information|Gathering Information]].
# Verify the physical connectivity between your device and end devices. See [[#Verifying Ports|Verifying Ports]].
# Verify the Layer 2 connectivity. See [[#Verifying Layer 2 Connectivity|Verifying Layer 2 Connectivity]].
# Verify the end-to-end connectivity and the routing configuration. See [[#Verifying Layer 3 Connectivity|Verifying Layer 3 Connectivity]].
# After you have determined that your troubleshooting attempts have not resolved the problem, contact Cisco TAC or your technical support representative.

{{note| View the [http://www.youtube.com/user/nxs7000 Cisco Nexus 7000 instructional videos] for an overview of Cisco NX-OS.}}

=== Gathering Information ===

This section describes the tools that are commonly used to troubleshoot problems within your network. Specific troubleshooting articles may include additional tools and commands specific to the symptoms and possible problems covered in that article.

{{note|You should have an accurate topology of your network to isolate problem areas. Contact your network architect for this information.}}

Use the following commands to gather general information on your device:
* '''show module'''
* '''show version'''
* '''show running-config'''
* '''show logging log'''
* '''show interfaces brief'''
* '''show vlan'''
* '''show spanning-tree '''
* '''show''' {'''ip''' | '''ipv6'''} '''routing'''
* '''show processes | include ER '''
* '''show accounting log'''

=== Verifying Ports ===

Answer the following questions to verify that your ports are connected correctly and are operational:

* Are you using the correct media (copper, optical, fiber type)?
* Is the media broken or damaged?
* Is the port LED green on the module?
* Is the interface in the correct VDC?
::Use the '''show vdc membership''' command to check which VDC that the interface is a member of. You must log into the device with the network-admin role to use this command.
* Is the interface operational?
::Use the '''show interface brief '''command. The status should be up.

See [[Cisco NX-OS Troubleshooting Guide, Release 4.0 -- Troubleshooting Ports|Troubleshooting Ports]] for more troubleshooting tips for ports.

=== Verifying Layer 2 Connectivity ===

Use the following commands to verify Layer 2 connectivity:
* Use the '''show vlan all-ports '''command to verify that all the necessary interfaces are in the same VLAN. The status should be active for the VLAN.
* Use the '''show port-channel compatibility-parameters''' command to verify that all the ports in a port channel are configured the same for the speed, the duplex, and the trunk mode.
* Use the '''show running-config spanning-tree command '''to verify that the Spanning Tree Protocol (STP) is configured the same on all devices in the network.
* Use the '''show processes | include ER''' command to verify that no essential Layer 2 processes are in the error state.
* Use the '''show spanning-tree blockedports '''command to display the ports that are blocked by STP.
* Use the '''show mac address-table dynamic vlan''' command to determine if learning or aging is occurring at each node.

See [[Cisco NX-OS Troubleshooting Guide, Release 4.0 -- Troubleshooting VLANs|Troubleshooting VLANs]] and [[Cisco NX-OS Troubleshooting Guide, Release 4.0 -- Troubleshooting STP|Troubleshooting STP]] for more information on troubleshooting Layer 2 issues.

=== Verifying Layer 3 Connectivity ===

Answer the following questions to verify Layer 3 connectivity:
* Have you configured a default gateway?
* Have you configured the same dynamic routing protocol parameters throughout your routing domain or configured static routes?
* Are any IP access lists, filters, or route maps blocking route updates?

Use the following commands to verify your routing configuration:
* '''show arp'''
* '''show ip routing '''
* '''show platform forwarding'''

See [[Cisco NX-OS Troubleshooting Guide, Release 4.0 -- Troubleshooting Tools and Methodology#Ping and Traceroute|Ping and Traceroute]] to verify Layer 3 connectivity. See [[Cisco NX-OS Troubleshooting Guide, Release 4.0 -- Troubleshooting Routing|Troubleshooting Routing]] for more information on troubleshooting Layer 3 issues.

== Overview of Symptoms ==

This article uses a symptom-based troubleshooting approach that allows you to diagnose and resolve your Cisco NX-OS problems by comparing the symptoms that you observed in your network with the symptoms listed in each chapter.

By comparing the symptoms in this publication to the symptoms that you observe in your own network, you should be able to diagnose and correct software configuration issues and inoperable hardware components so that the problems are resolved with minimal disruption to the network. Those problems and corrective actions include the following:

*Identify key Cisco NX-OS troubleshooting tools.
*Obtain and analyze protocol traces using SPAN and RSPAN or Ethanalyzer on the CLI.
*Identify or rule out physical port issues.
*Identify or rule out switch module issues.
*Diagnose and correct Layer 2 issues.
*Diagnose and correct Layer 3 issues.
*Recover from switch upgrade failures.
*Obtain core dumps and other diagnostic data for use by Cisco TAC or your customer support representative.

== System Messages ==

The system software sends syslog (system) messages to the console (and, optionally, to a logging server on another device). Not all messages indicate a problem with your device. Some messages are purely informational, while others might help diagnose problems with links, internal hardware, or the device software.

=== System Message Text ===

Message text is a text string that describes the condition. This portion of the message might contain detailed information about the event, including terminal port numbers, network addresses, or addresses that correspond to locations in the system memory address space. Because the information in these variable fields changes from message to message, it is represented here by short strings enclosed in square brackets (<nowiki>[ ]</nowiki>). A decimal number, for example, is represented as <nowiki>[dec ]</nowiki>.

:<nowiki>PORT-3-IF_UNSUPPORTED_TRANSCEIVER: Transceiver for interface [chars] is not supported.</nowiki>

Use this string to find the matching system message in the [http://www.cisco.com/en/US/docs/switches/datacenter/sw/system_messages/reference/sl_nxos_book.html|Cisco NX-OS System Messages Reference].

Each system message is followed by an explanation and recommended action. The action may be as simple as "No action is required." It may involve a fix or a recommendation to contact technical support as shown in the following example:

::'''Error Message''' PORT-3-IF_UNSUPPORTED_TRANSCEIVER: Transceiver for interface <nowiki>[chars] is not supported.</nowiki>

::'''Explanation''' Transceiver (SFP) is not from an authorized vendor.

::'''Recommended Action''' Enter the '''show interface transceiver''' CLI command or similar DCNM command to determine the transceiver being used. Please contact your customer support representative for a list of authorized transceiver vendors.

=== syslog Server Implementation ===

The syslog facility allows the Cisco NX-OS device to send a copy of the message log to a host for more permanent storage. This feature allows you to examine the logs over a long period of time or if the Cisco NX-OS device is not accessible.

This example shows how to configure a Cisco NX-OS device to use the syslog facility on a Solaris platform. Although a Solaris host is being used, the syslog configuration on all UNIX and Linux systems is very similar.

syslog uses the facility to determine how to handle a message on the syslog server (the Solaris system in this example) and the message severity. Different message severities are handled differently by the syslog server. They could be logged to different files or e-mailed to a particular user. Specifying a severity level on the syslog server determines that all messages of that level and greater severity (lower number) will be acted upon as you configure the syslog server.

{{note|You should configure the syslog server so that the Cisco NX-OS messages are logged to a different file from the standard syslog file so that they cannot be confused with other non-Cisco syslog messages. Do not locate the logfile on the / file system. You do not want log messages to fill up the / file system. This example uses the following values:
:*syslog client: switch1
:*syslog server: 172.22.36.211
:*(Solaris) syslog facility: local1
:*syslog severity: notifications (level 5, the default)
:*File to log Cisco NX-OS messages to: /var/adm/nxos_logs}}

To configure the syslog feature on Cisco NX-OS, follow these steps:
# switch1# '''config terminal'''
# switch1(config)# '''logging server 192.0.2.1 6 facility local1'''

Use the '''show logging server '''command to verify the syslog configuration.
:switch1# '''show logging server'''
<pre>Logging server: enabled
{172.22.36.211}
server severity: notifications
server facility: local1
server VRF: management</pre>

To configure a syslog server, follow these steps:

1. Modify /etc/syslog.conf to handle local1 messages. For Solaris, you must allow at least one tab between the facility.severity and the action (/var/adm/nxos_logs).
: '''local1.notice /var/adm/nxos_logs'''

2. Create the log file.
: '''touch /var/adm/nxos_logs'''

3. Restart the syslog process.
: '''/etc/init.d/syslog stop'''
: '''/etc/init.d/syslog start'''
<pre>syslog service starting.</pre>

4. Verify that the syslog process has started.
: '''ps -ef |grep syslogd'''
<pre>root 23508 1 0 11:01:41 ? 0:00 /usr/sbin/syslogd</pre>

Test the syslog server by creating an event in Cisco NX-OS. In this case, port e1/2 was shut down and reenabled and the following was listed on the syslog server. The IP address of the switch is listed in brackets.
:'''tail -f /var/adm/MDS_logs'''
<pre>Sep 17 11:07:41 [172.22.36.142.2.2] : 2004 Sep 17 11:17:29 pacific: PORT-5-IF_DOWN_INITIALIZING: %$VLAN 1%$ Interface e 1/2 is down (Initializing)

Sep 17 11:07:49 [172.22.36.142.2.2] : 2004 Sep 17 11:17:36 pacific: %PORT-5-IF_UP: %$VLAN 1%$ Interface e 1/2 is up in mode access

Sep 17 11:07:51 [172.22.36.142.2.2] : 2004 Sep 17 11:17:39 pacific: %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configuring console from pts/0 (dhcp-171-71-49-125.cisco.com </pre>

==Troubleshooting with Logs==

Cisco NX-OS generates many types of system messages on the device and sends them to a syslog server. You can view these messages to determine what events may have led up to the current problem condition that you are facing.

Use the following commands to access and view logs in Cisco NX-OS:

switch# '''show logging ?'''

<pre>console Show console logging configuration
info Show logging configuration
internal syslog syslog internal information
ip IP configuration
last Show last few lines of logfile
level Show facility logging configuration
logfile Show contents of logfile
loopback Show logging loopback configuration
module Show module logging configuration
monitor Show monitor logging configuration
nvram Show NVRAM log
onboard show logging onboard
pending server address pending configuration
pending-diff server address pending configuration diff
server Show server logging configuration
session Show logging session status
status Show logging status
timestamp Show logging timestamp configuration</pre>

This example shows the output of the '''show logging server''' command:

switch# '''show logging server'''
<pre>Logging server: enabled
{172.28.254.254}
server severity: notifications
server facility: local7
server VRF: management</pre>

==Troubleshooting Modules ==

You can directly connect to a module console port to troubleshoot module bootup issues.
Use the '''attach console module''' command to connect to the module console port.

==Viewing NVRAM logs==

System messages that are priority 0, 1, or 2 are logged into NVRAM on the supervisor module. After a switch reboots, you can display these syslog messages in NVRAM by using the '''show logging nvram''' command.

switch# '''show logging nvram'''
<pre>2008 Sep 10 15:51:58 switch %$ VDC-1 %$ %SYSMGR-2-NON_VOLATILE_DB_FULL: System n
on-volatile storage usage is unexpectedly high at 99%.
2008 Sep 10 15:52:13 switch %$ VDC-1 %$ %PLATFORM-2-PFM_SYSTEM_RESET: Manual sys
tem restart from Command Line Interface
2008 Sep 10 15:57:49 switch %$ VDC-1 %$ %KERN-2-SYSTEM_MSG: Starting kernel... -
kernel
2008 Sep 10 15:58:00 switch %$ VDC-1 %$ %CARDCLIENT-2-REG: Sent
2008 Sep 10 15:58:01 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: R2D2: P1 SUP NO GMTL
FOR P1 SUP - r2d2
2008 Sep 10 15:58:01 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: R2D2: P1 SUP NO GMTL
FOR P1 SUP - r2d2
2008 Sep 10 15:58:05 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: R2D2: P1 SUP: Reset
Tx/Rx during QOS INIT - r2d2
2008 Sep 10 15:58:16 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: can't dlsym ssnmgr_i
s_session_command: please link this binary with ssnmgr.so! - svi
2008 Sep 10 15:58:16 switch %$ VDC-1 %$ %CARDCLIENT-2-SSE: LC_READY sent
2008 Sep 10 15:58:17 switch %$ VDC-1 %$ snmpd: load_mib_module :Error, while loa
ding the mib module /isan/lib/libpmsnmp_common.so (/isan/lib/libpmsnmp_common.so
: undefined symbol: sme_mib_get_if_info)
2008 Sep 10 15:58:17 switch %$ VDC-1 %$ %CARDCLIENT-2-SSE: MOD:6 SUP ONLINE
2008 Sep 10 15:58:17 switch %$ VDC-1 %$ %VDC_MGR-2-VDC_LIC_WARN: Service using g
race period will be shutdown in 9 day(s)</pre>

== Contacting Customer Support ==

If you are unable to solve a problem after using the troubleshooting suggestions in these articles, contact a customer service representative for assistance and further instructions. Before you call, have the following information ready to help your service provider assist you as quickly as possible:

*Date that you received the switch
*Chassis serial number (located on a label on the right side of the rear panel of the chassis)
*Type of software and release number
*Maintenance agreement or warranty information
*Brief description of the problem
*Brief explanation of the steps that you have already taken to isolate and resolve the problem

For more information on steps to take before calling Technical Support, see [[Cisco NX-OS Troubleshooting Guide, Release 4.0 -- Before Contacting Technical Support|Before Contacting Technical Support]].

== See Also ==
[[Cisco NX-OS Troubleshooting Guide, Release 4.0 -- Before Contacting Technical Support|Before Contacting Technical Support]]

== Further Reading ==
The following links contain further information on this topic from Cisco.com:

[http://www.cisco.com/en/US/docs/switches/datacenter/sw/system_messages/reference/sl_nxos_book.html Cisco NX-OS System Messages Reference]

== External Links ==

External links contain content developed by external authors. Cisco does not review this content for accuracy.

[http://www.networkworld.com/community/node/26877 Nexus: Hands on with NX-OS, Part#1]

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide

2011-11-30T21:32:53Z

Sukumars:

{{Template:Required Metadata}}
This article provides guidance for troubleshooting issues that may appear when using Cisco Nexus 7000 Series. This article introduces tools and methodologies to recognize a problem, determine its cause, and find possible solutions. However, these documentation helps only in basic troubleshooting.

''''''We encourage uses to review the Cisco Live Presentation for detailed troubleshooting for Nexus 7000.

Sections of this presentation covers, both platform independent, and platform specific step by step
troubleshooting for most common issues. Access to this presentation is available FREE. Follow the below instructions to access the
presentation.

1. Visit https://www.ciscolivevirtual.com/

2. Register for free.

3. Click on "Cisco Live Virtual" link.

4. Click on the ‘Sessions’ Tab on top, and select ‘2011 Sessions Catalog’

5. In the search box, type “BRKCRS-3144” and Submit search.

6. Select the session. You can either View the Session (or) download the pdf'''
'''

{| align="center" border="1"
|align="left"|Welcome to '''Cisco DocWiki'''. We encourage [http://tools.cisco.com/RPF/register/register.do registered Cisco.com users] to contribute to this wiki to improve Cisco product documentation. Note that you cannot log in to DocWiki with Cisco.com "guest" account credentials. 
See [[DocWiki:Terms_of_use|Terms of Use]] and [[DocWiki:About|About DocWiki]] for more information about Cisco DocWiki. 
Select the "edit" tab to edit an article or select the "discussion" tab to submit questions or comments about the article. 

Click [http://www.cisco.com/en/US/products/ps9402/prod_troubleshooting_guides_list.html here] to return to the Cisco Nexus 7000 Series documentation on www.cisco.com.
|}

__TOC__

==Audience and Generating a PDF of This Guide==

This article is for experienced network administrators who configure and maintain NX-OS devices.

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Book PDF]]

==Organization==

This article is organized into the following sections:

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Overview|Troubleshooting Overview]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Installs, Upgrades, and Reboots|Troubleshooting Installs, Upgrades, and Reboots]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Licensing|Troubleshooting Licensing]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VDCs|Troubleshooting VDCs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting CFS|Troubleshooting CFS]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Ports|Troubleshooting Ports]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs|Troubleshooting vPCs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VLANs|Troubleshooting VLANs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting STP|Troubleshooting STP]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Routing|Troubleshooting Routing]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Unicast Traffic|Troubleshooting Unicast Traffic]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Multicast Traffic|Troubleshooting Multicast Traffic]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting WCCP|Troubleshooting WCCP]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Memory|Troubleshooting Memory]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Packet Flow Issues|Troubleshooting Packet Flow Issues]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Before Contacting Technical Support|Before Contacting Technical Support]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Tools and Methodology|Troubleshooting Tools and Methodology]]

[[Category:Nexus 7000 Series Switches]]

[[Category:Troubleshooting]]

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide

2011-11-30T21:31:06Z

Sukumars:

{{Template:Required Metadata}}
This article provides guidance for troubleshooting issues that may appear when using Cisco Nexus 7000 Series. This article introduces tools and methodologies to recognize a problem, determine its cause, and find possible solutions. However, these documentation helps only in basic troubleshooting.

We encourage uses to review the Cisco Live Presentation for detailed troubleshooting for Nexus 7000.

Sections of this presentation covers, both platform independent, and platform specific step by step
troubleshooting for most common issues. Access to this presentation is available FREE. Follow the below instructions to access the
presentation.

1. Visit https://www.ciscolivevirtual.com/

2. Register for free.

3. Click on "Cisco Live Virtual" link.

4. Click on the ‘Sessions’ Tab on top, and select ‘2011 Sessions Catalog’

5. In the search box, type “BRKCRS-3144” and Submit search.

6. Select the session. You can either View the Session (or) download the pdf

{| align="center" border="1"
|align="left"|Welcome to '''Cisco DocWiki'''. We encourage [http://tools.cisco.com/RPF/register/register.do registered Cisco.com users] to contribute to this wiki to improve Cisco product documentation. Note that you cannot log in to DocWiki with Cisco.com "guest" account credentials. 
See [[DocWiki:Terms_of_use|Terms of Use]] and [[DocWiki:About|About DocWiki]] for more information about Cisco DocWiki. 
Select the "edit" tab to edit an article or select the "discussion" tab to submit questions or comments about the article. 

Click [http://www.cisco.com/en/US/products/ps9402/prod_troubleshooting_guides_list.html here] to return to the Cisco Nexus 7000 Series documentation on www.cisco.com.
|}

__TOC__

==Audience and Generating a PDF of This Guide==

This article is for experienced network administrators who configure and maintain NX-OS devices.

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Book PDF]]

==Organization==

This article is organized into the following sections:

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Overview|Troubleshooting Overview]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Installs, Upgrades, and Reboots|Troubleshooting Installs, Upgrades, and Reboots]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Licensing|Troubleshooting Licensing]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VDCs|Troubleshooting VDCs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting CFS|Troubleshooting CFS]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Ports|Troubleshooting Ports]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs|Troubleshooting vPCs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VLANs|Troubleshooting VLANs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting STP|Troubleshooting STP]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Routing|Troubleshooting Routing]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Unicast Traffic|Troubleshooting Unicast Traffic]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Multicast Traffic|Troubleshooting Multicast Traffic]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting WCCP|Troubleshooting WCCP]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Memory|Troubleshooting Memory]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Packet Flow Issues|Troubleshooting Packet Flow Issues]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Before Contacting Technical Support|Before Contacting Technical Support]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Tools and Methodology|Troubleshooting Tools and Methodology]]

[[Category:Nexus 7000 Series Switches]]

[[Category:Troubleshooting]]

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs

2011-11-30T21:27:06Z

Sukumars:

{{Template:Required Metadata}}

This article describes how to do basic troubleshooting of virtual Port Channel(vPC) problems on a Cisco Nexus 7000 NX-OS device.

Troubleshooting vPC on Nexus 7000 is covered in detail in Cisco-Live presentation. Sections of this presentation covers,
both platform independent, and platform specific step by step troubleshooting for vPC, among other things. Access to
this presentation is available FREE. Follow the below instructions to access the presentation

1. Visit https://www.ciscolivevirtual.com/

2. Register for free.

3. Click on "Cisco Live Virtual" link.

4. Click on the ‘Sessions’ Tab on top, and select ‘2011 Sessions Catalog’

5. In the search box, type “BRKCRS-3144” and Submit search.

6. Select the session. You can either View the Session (or) download the pdf.

7. vPC troubleshooting is covered from slides 64 through 80.

{| align="right" border="1"
|aling="center"|'''Guide Contents'''
|-
|[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Overview|Troubleshooting Overview]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Installs, Upgrades, and Reboots|Troubleshooting Installs, Upgrades, and Reboots]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Licensing|Troubleshooting Licensing]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VDCs|Troubleshooting VDCs]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting CFS|Troubleshooting CFS]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Ports|Troubleshooting Ports]] ''Troubleshooting vPCs {this section}'' [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VLANs|Troubleshooting VLANs]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting STP|Troubleshooting STP]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Routing|Troubleshooting Routing]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Unicast Traffic|Troubleshooting Unicast Traffic]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting WCCP|Troubleshooting WCCP]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Memory|Troubleshooting Memory]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Packet Flow Issues|Troubleshooting Packet Flow Issues]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Before Contacting Technical Support|Before Contacting Technical Support]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Tools and Methodology|Troubleshooting Tools and Methodology]]
|}

__TOC__

== Information About Troubleshooting vPCs ==

A vPC allows links that are physically connected to two different Cisco Nexus 7000 Series devices to appear as a single port channel by a third device.

See the [http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_1/nx-os/interfaces/configuration/guide/if_vPC.html Configuring vPC chapter] in the [http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_1/nx-os/interfaces/configuration/guide/if_nxos.html Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide] for more information on vPCs.

== Initial Troubleshooting Checklist ==

Begin troubleshooting vPC issues by checking the following issues first:

{| border = 1
|-
!Checklist
!Check off
|-
|

Verify that all vPC interfaces in a vPC domain are configured in the same virtual device context (VDC).
|
|-
|

Verify that you have a separate vPC peer-link and peer-keepalive link infrastructure for each VDC deployed.
|
|-
|

Is the vPC keepalive link mapped to a separate vrf? If not, it will be mapped to the management vrf by default. In this case, do you have a management switch connect to the management ports on both vPC peer devices?
|
|-
|

Verify that the vPC peer-link is configured on a N7K-M132XP-12. It is recommended to have at least two N7K-M132XP-12 for redundancy.
|
|-
|

Verify that both the source and destination IP addresses used for the peer-keepalive messages are reachable from the VRF associated with the vPC peer-keepalive link.
|
|-
|

Verify that the peer-keepalive link is up or the vPC peer-link will not come up.
|
|-
|

Verify that the vPC peer-link is configured as a Layer 2 Port Channel trunk which only allows vPC VLANs.
|
|-
|

Verify that the vPC number that you assigned to the port channel that connects to the downstream device from the vPC peer device is identical on both vPC peer devices.
|
|-
|

If you manually configured the system priority, verify that you assigned the same priority value on both vPC peer devices.
|
|-
|

Check the '''show vpc consistency-parameters''' command to verify that both vPC peer devices have identical type-1 parameters.
|
|-
|

Verify that the primary vPC is the primary STP root and the secondary vPC is the secondary STP root.
|
|-
|}

== Verifying vPCs Using the CLI ==

To verify vPCs using the CLI, follow these steps:

1. Use the '''show running-config vpc''' command to verify the vPC configuration.

2. Use the '''show vpc ''' command to check the status of vPC.

3. Use the '''show vpc peer-keepalive''' command to check the status of the vPC peer-keepalive link.

4. Use the '''show vpc consistency-parameters''' command to verify that both the vPC peers have the identical type-1 parameters.

5. Use the '''show port-channel summary''' command toverify the members in the port channel are mapped to the vPC.

6. Use the '''show cfs status''' commands to verify that distribution over Ethernet is enabled.

7. If you enable STP, use the '''show spanning-tree''' command on both sides of the vPC peer link to verify that the following STP parameters are identical:

* BPDU Filter
* BPDU Guard
* Cost
* Link type
* Priority
* VLANs (PVRST+)

== Received Type 1 Configuration Element Mismatch ==

You may have a problem where you cannot bring up a vPC link because of a type 1 configuration element mismatch.

{| border = 1
|-
!Symptom
!Possible Cause
!Solution
|-
|

Received a type 1 configuration element mismatch.
|

The vPC peer ports or membership ports do not have identical configurations.
|

Use the ''' show vpc consistency-parameters interface''' command to determine where the configuration mismatch occurs.
|
|}

==== Example: show vpc consistency-parameters Command Output====

This example shows how to display the vPC consistency parameters on a port channel:

switch# '''show vpc consistency-parameters interface po 10'''
Legend:
Type 1 : vPC will be suspended in case of mismatch
Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
STP Mode 1 Rapid-PVST Rapid-PVST
STP Disabled 1 None None
STP MST Region Name 1 "" ""
STP MST Region Revision 1 0 0
STP MST Region Instance to 1
VLAN Mapping
STP Loopguard 1 Disabled Disabled
STP Bridge Assurance 1 Enabled Enabled
STP Port Type 1 Normal Normal
STP MST Simulate PVST 1 Enabled Enabled
Allowed VLANs - 1-10,15-20,30,37,99 1-10,15-20,30,37,99

== Cannot Enable the vPC Feature==

You may receive an error when you enable the vPC feature.

{| border = 1
|-
!Symptom
!Possible Cause
!Solution
|-
|

Cannot enable the vPC feature.
|

The hardware is incompatible with the vPC.
|

Use the ''' show module ''' command to determine the hardware version of each N7K-M132XP-12 Ethernet module. The hardwere version must be 1.3 or later to enable the vPC feature.
|}

==== Example: show module Command Output====

This example shows how to display the module hardware version:

switch# '''show module'''
Mod Ports Module-Type Model Status
--- ----- -------------------------------- ------------------ ------------
2 32 10 Gbps Ethernet Module N7K-M132XP-12 ok
3 48 10/100/1000 Mbps Ethernet Module N7K-M148GT-11 ok
5 0 Supervisor module-1X N7K-SUP1 active *
6 0 Supervisor module-1X N7K-SUP1 ha-standby
10 32 10 Gbps Ethernet Module N7K-M132XP-12 ok

Mod Sw Hw
--- -------------- ------
2 4.1(5) 1.2
3 4.1(5) 1.0 '''>>> Must be 1.3 or later.'''

== vPC in Blocking State==

vPC may be in the blocking state because of Bride Assurance (BA).

{| border = 1
|-
!Symptom
!Possible Cause
!Solution
|-
|

vPC is in blocking state.
|

BPDU only sends on a single link of a port-channel. If BA dispute is detected, the entire vPC will be in the blocking state.
|

Do not enable BA on vPC.
|}

== VLANs on a vPC moved to suspend state==

VLANs on a vPC may move to the suspend state.

{| border = 1
|-
!Symptom
!Possible Cause
!Solution
|-
|

VLANs on a vPC moved to suspend state.
|

VLANs allowed on the vPC have not been allowed on the vPC peer=link.
|

All VLANs allowed on a vPC must also be allowed on the vPC peer-link. Also, it is recommended that only vPC VLANs are allowed on the vPC peer-link.
|}

== Hosts with an HSRP Gateway Cannot Access Beyond Their VLAN ==

When HSRP is enabled on both vPC peer devices on a VLAN and hosts on that VLAN set the HSRP as their gateway, they may not able to reach anything outside their own VLAN.

{| border = 1
|-
!Symptom
!Possible Cause
!Solution
|-
|

Hosts with an HSRP gateway cannot access beyond their VLAN.
|

If the host gateway mac-address is mapped to the physical MAC address of any one of the vPC peer-devices, packets may get dropped due to the loop prevention mechanism in vPC.
|

Map the host gateway's mac-address to the HSRP MAC address and not the physical MAC address of any one of the vPC peer-devices. Peer-gateway can be a workaround for this scenario. Please read the configuration guide for peer-gateway for further information before implementing it.
|}

== Traffic Disrupted when the Primary vPC Device Goes Down ==

Traffic may remain disrupted when the N7K-M132XP-12 module on the primary vpc device goes down.

{| border = 1
|-
!Symptom
!Possible Cause
!Solution
|-
|

Traffic disrupted when the primary vPC device goes down.
|

All core facing interfaces and vPC peer-links are configured on a single N7K-M132XP-12 module.
|

Enable object tracking. With object tracking enabled, all vPC on the primary will shut down. The vPC secondary will take over as the operational primary and all the vPC on the secondary will stay up. As a result, traffic will still be flowing thru the secondary which became operational primary.
|}

== See Also ==
[[Cisco NX-OS Troubleshooting Guide, Release 4.0 -- Troubleshooting Ports|Troubleshooting Ports]]

== Further Reading ==
The following links contain further information on this topic from Cisco.com:

[http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11-516396.html Virtual PortChannels: Building Networks without Spanning Tree Protocol (Cisco White Paper)]

[http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/dcnm/interfaces/configuration/guide/if_vPC.html Configuring vPCs (Cisco Nexus 7000 Series Interfaces Configuration Guide)]

[http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/configuration_guide_c07-543563.html Nexus 5000 Virtual PortChannel Quick Configuration Guide]

== External Links ==

The following links contain content developed by external authors. Cisco does not review this content for accuracy.

[http://www.networkworld.com/community/node/38325 Our Nexus Data Center Network - To vPC or not to vPC]

[http://shouldhavegonewithcisco.com/2009/03/16/nexus-7000-virtual-portchannel-part-1/ Nexus 7000 Virtual Portchannel Part 1]

[http://shouldhavegonewithcisco.com/2009/03/30/nexus-7000-virtual-portchannel-part-2/ Nexus 7000 Virtual Portchannel Part 2]

[http://shouldhavegonewithcisco.com/2009/04/30/nexus-7000-virtual-portchannel-part-3-final/ Nexus 7000 Virtual Portchannel Part 3]

[http://jasonnash.wordpress.com/2009/08/10/vpc-virtual-port-channel-and-the-nexus-platform/ vPC (Virtual Port-Channel) and the Nexus 5000 Platform]

[http://www.howfunky.com/search/label/Cisco%20Nexus Blog on Cisco Nexus Features (VLANs, vPCs)]

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Unicast Traffic

2011-11-30T21:23:36Z

Sukumars:

{{Template:Required Metadata}}

This article below provides only basic information on how to troubleshoot unicast packet flow traffic issues for the M1 Series modules.

Troubleshooting L2/L3 unicast is covered in detail in Cisco-Live presentation. Sections of this presentation covers,
both platform independent, and platform specific step by step troubleshooting for unicast, among other things. Access to
this presentation is available FREE. Follow the below instructions to access the presentation

1. Visit https://www.ciscolivevirtual.com/

2. Register for free.

3. Click on "Cisco Live Virtual" link.

4. Click on the ‘Sessions’ Tab on top, and select ‘2011 Sessions Catalog’

5. In the search box, type “BRKCRS-3144” and Submit search.

6. Select the session. You can either View the Session (or) download the pdf.

7. Unicast troubleshooting is covered from slides 81 through 93.

{| align="right" border="1"
|align="center"|'''Guide Contents'''
|-
|[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Overview|Troubleshooting Overview]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Installs, Upgrades, and Reboots|Troubleshooting Installs, Upgrades, and Reboots]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Licensing|Troubleshooting Licensing]]' [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VDCs|Troubleshooting VDCs]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting CFS|Troubleshooting CFS]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Ports|Troubleshooting Ports]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs|Troubleshooting vPCs]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VLANs|Troubleshooting VLANs]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting STP|Troubleshooting STP]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Routing|Troubleshooting Routing]] ''Troubleshooting Unicast Traffic(this section)'' [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting WCCP|Troubleshooting WCCP]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Memory|Troubleshooting Memory]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Packet Flow Issues|Troubleshooting Packet Flow Issues]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Before Contacting Technical Support|Before Contacting Technical Support]] [[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Tools and Methodology|Troubleshooting Tools and Methodology]]
|}

__TOC__

==Packet is Received into Interface from Wire==

During this step, the packet is received into the Nexus 7000 port. When troubleshooting this step, we want to look to ensure there is transceiver interoperability, and validate whether we are seeing any errors on the interface. We do this via using the following commands

* '''show interface''' ''interface''
* '''show interface''' ''interface'' '''transceiver'''

PHX2-N7K-1# '''show interface e1/1'''
'''Ethernet1/1 is up'''
Hardware: 10000 Ethernet, address: 0024.986c.00b0 (bia 0024.986c.00b0)
Description: N7K-vdc-1 connecting to core 6506
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s, media type is 10g
Beacon is turned off
Auto-Negotiation is turned off
Input flow-control is off, output flow-control is off
'''Rate mode is shared'''
Switchport monitor is off
Last link flapped 7week(s) 4day(s)
Last clearing of "show interface" counters never
'''1 minute input rate''' 13056 bits/sec, 9 packets/sec
'''1 minute output rate''' 4608 bits/sec, 0 packets/sec
Rx
341190251 input packets 276211313 unicast packets 52112947 multicast packets
12865991 broadcast packets 0 jumbo packets 0 storm suppression packets
94295027129 bytes
Tx
462437316 output packets 85121 multicast packets
188251 broadcast packets 0 jumbo packets
648159081064 bytes
'''0 input error 0 short frame 0 watchdog
0 no buffer 0 runt 0 CRC 0 ecc
0 overrun 0 underrun 0 ignored 0 bad etype drop
0 bad proto drop 0 if down drop 0 input with dribble
0 input discard
0 output error 0 collision 0 deferred
0 late collision 0 lost carrier 0 no carrier
0 babble
0 Rx pause 0 Tx pause'''
1 interface resets

PHX2-N7K-1# '''show interface e1/1 transceiver details '''
Ethernet1/1
sfp is present
name is CISCO-AVAGO '''<<< If this says type is (unknown), it is not supported.'''
part number is SFBR-7700SDZ
revision is B4
serial number is AGD12434116
nominal bitrate is 10300 MBits/sec
Link length supported for 50/125um fiber is 82 m(s)
Link length supported for 62.5/125um fiber is 26 m(s)
cisco id is --
cisco extended id number is 4

SFP Detail Diagnostics Information (internal calibration)
----------------------------------------------------------------------------
Alarms Warnings
High Low High Low
----------------------------------------------------------------------------
Temperature 45.46 C 75.00 C -5.00 C 70.00 C 0.00 C
Voltage 3.28 V 3.63 V 2.97 V 3.46 V 3.13 V
Current 6.92 mA 10.50 mA 2.50 mA 10.50 mA 2.50 mA
Tx Power -2.75 dBm 1.69 dBm -11.30 dBm -1.30 dBm -7.30 dBm
Rx Power -2.43 dBm 1.99 dBm -13.97 dBm -1.00 dBm -9.91 dBm
Transmit Fault Count = 0
----------------------------------------------------------------------------
Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning

==Linksec Decryption Occurs, 1st stage Port QoS==

In step 2, Linksec decryption occurs as well as receive side stage 1 QoS.

It is important to step back and evaluate the difference between stage 1 and stage 2 QoS. The difference is that some ports can be configured in shared mode, whereas some can be configured in dedicated mode, on the 10G modules. What this means, is that there is 10g of bandwidth that can be dedicated to a port or shared amongst ports (4 ports, on the m132 module).

When running in shared mode, there exists a chance for contention accessing the 10g bandwidth through the 4:1 Mux. To alleviate this, some QoS intelligence was passed down to the 4:1 Mux which aggregates the ports.

In dedicated mode, there is no QoS applied at the Mux, instead, all traffic is processed in phase 2 QoS. To summarize, in shared mode, 1st stage QoS ensures fair access to the 10g of port bandwidth. In both shared and dedicated mode, 2nd stage QoS occurs to provide ingress queuing to the system.

For the ingress QoS, we are concerned about the Receive side QoS parameters in the show queuing command.

Use the '''show policy-map''' command to see per queue dropped packets.

The commands to troubleshoot Linksec and Port QoS are as follows:

* '''show cts interface''' [''all'' | ''interface'']
* '''show queuing interface''' ''interface''
* '''show policy-map interface''' (for per queue drop)

switch# '''show cts interface all Working Example'''
CTS Information for Interface Ethernet2/24:
CTS is enabled, mode: CTS_MODE_DOT1X
IFC state: CTS_IFC_ST_CTS_OPEN_STATE
Authentication Status: CTS_AUTHC_SUCCESS
Peer Identity: india1
Peer is: CTS Capable
802.1X role: CTS_ROLE_AUTH
Last Re-Authentication:
Authorization Status: CTS_AUTHZ_SUCCESS
PEER SGT: 2
Peer SGT assignment: Trusted
Global policy fallback access list:
SAP Status: '''CTS_SAP_SUCCESS'''
Configured pairwise ciphers: GCM_ENCRYPT
Replay protection: Enabled
Replay protection mode: Strict
Selected cipher: GCM_ENCRYPT
Current receive SPI: '''sci:1b54c1fbff0000 an:0'''
Current transmit SPI: '''sci:1b54c1fc000000 an:0'''

PHX2-N7K-1# '''show cts interface eth 1/8 Broken Example'''
CTS Information for Interface Ethernet1/8:
CTS is enabled, mode: CTS_MODE_MANUAL
IFC state: Unknown
Authentication Status: CTS_AUTHC_INIT
Peer Identity:
Peer is: '''Not CTS Capable'''
802.1X role: CTS_ROLE_UNKNOWN
Last Re-Authentication:
Authorization Status: CTS_AUTHZ_INIT
PEER SGT: 0
Peer SGT assignment: Not Trusted
SAP Status: CTS_SAP_INIT
Configured pairwise ciphers:
Replay protection:
Replay protection mode:
Selected cipher:
Current receive SPI:
Current transmit SPI:

PHX2-N7K-1# '''show queuing int eth 1/1'''
Interface Ethernet1/1 TX Queuing strategy: Weighted Round-Robin
Port QoS is enabled
Queuing Mode in TX direction: mode-cos
Transmit queues [type = 1p7q4t]
Queue Id Scheduling Num of thresholds
_____________________________________________________________
1p7q4t-out-q-default WRR 04
1p7q4t-out-q2 WRR 04
1p7q4t-out-q3 WRR 04
1p7q4t-out-q4 WRR 04
1p7q4t-out-q5 WRR 04
1p7q4t-out-q6 WRR 04
1p7q4t-out-q7 WRR 04
1p7q4t-out-pq1 Priority 04

Configured WRR
WRR bandwidth ratios: 25[1p7q4t-out-q-default] 15[1p7q4t-out-q2] 12[1p7q4t-out-q3]
12[1p7q4t-out-q4] 12[1p7q4t-out-q5] 12[1p7q4t-out-q6] 12[1p7q4t-out-q7]
WRR configuration read from HW
WRR bandwidth ratios: 25[1p7q4t-out-q-default] 15[1p7q4t-out-q2] 11[1p7q4t-out-q3]
11[1p7q4t-out-q4] 11[1p7q4t-out-q5] 11[1p7q4t-out-q6] 11[1p7q4t-out-q7]

Configured queue-limit ratios
queue-limit ratios: 78[1p7q4t-out-q-default] 1[1p7q4t-out-q2] 1[1p7q4t-out-q3]
*1[1p7q4t-out-q4] *1[1p7q4t-out-q5] *1[1p7q4t-out-q6] *1[1p7q4t-out-q7] 16[1p7q4t-out-pq1]
* means unused queue with mandatory minimum queue-limit

queue-limit ratios configuration read from HW
queue-limit ratios: 78[1p7q4t-out-q-default] 1[1p7q4t-out-q2] 1[1p7q4t-out-q3]
*1[1p7q4t-out-q4] *1[1p7q4t-out-q5] *1[1p7q4t-out-q6] *1[1p7q4t-out-q7] 16[1p7q4t-out-pq1]
* means unused queue with mandatory minimum queue-limit

Thresholds:
COS Queue Threshold Type Min Max
__________________________________________________________________
0 1p7q4t-out-q-default DT 100 100
1 1p7q4t-out-q-default DT 100 100
2 1p7q4t-out-q-default DT 100 100
3 1p7q4t-out-q-default DT 100 100
4 1p7q4t-out-q-default DT 100 100
5 1p7q4t-out-pq1 DT 100 100
6 1p7q4t-out-pq1 DT 100 100
7 1p7q4t-out-pq1 DT 100 100

'''Interface Ethernet1/1 RX Queuing strategy: Weighted Round-Robin'''
'''Queuing Mode in RX direction: mode-cos'''
'''Receive queues [type = 8q2t]'''
''' Port Cos not configured'''
Queue Id Scheduling Num of thresholds
____________________________________________________________
8q2t-in-q-default WRR 02
8q2t-in-q2 WRR 02
8q2t-in-q3 WRR 02
8q2t-in-q4 WRR 02
8q2t-in-q5 WRR 02
8q2t-in-q6 WRR 02
8q2t-in-q7 WRR 02
8q2t-in-q1 WRR 02

Configured WRR
WRR bandwidth ratios: 20[8q2t-in-q-default] 0[8q2t-in-q2] 0[8q2t-in-q3]
0[8q2tin-q4] 0[8q2t-in-q5] 0[8q2t-in-q6] 0[8q2t-in-q7] 80[8q2t-in-q1]
WRR configuration read from HW
WRR bandwidth ratios: 20[8q2t-in-q-default] 0[8q2t-in-q2] 0[8q2t-in-q3]
0[8q2t-in-q4] 0[8q2t-in-q5] 0[8q2t-in-q6] 0[8q2t-in-q7] 80[8q2t-in-q1]

No queue-limit ratios user configuration
________________________________________

queue-limit ratios configuration read from HW
queue-limit ratios: 100[8q2t-in-q-default] 100[8q2t-in-q2] 100[8q2t-in-q3]
100[8q2t-in-q4] 100[8q2t-in-q5] 100[8q2t-in-q6] 100[8q2t-in-q7] 100[8q2t-in-q1]

Thresholds:
COS Queue Threshold Type Min Max
__________________________________________________________________
0 8q2t-in-q-default DT 100 100
1 8q2t-in-q-default DT 100 100
2 8q2t-in-q-default DT 100 100
3 8q2t-in-q-default DT 100 100
4 8q2t-in-q-default DT 100 100
5 8q2t-in-q1 DT 100 100
6 8q2t-in-q1 DT 100 100
7 8q2t-in-q1 DT 100 100

PHX2-N7K-1# '''show policy-map interface eth 1/2'''
Global statistics status : enabled
Ethernet1/2
Service-policy (queuing) '''input:''' default-in-policy
policy statistics status: enabled

Class-map (queuing): in-q1 (match-any)
queue-limit percent 50
bandwidth percent 80
'''queue dropped pkts : 0 '''

Class-map (queuing): in-q-default (match-any)
queue-limit percent 50
bandwidth percent 20
'''queue dropped pkts : 0'''

Service-policy (queuing) '''output: ''' default-out-policy
policy statistics status: enabled

Class-map (queuing): out-pq1 (match-any)
priority level 1
queue-limit percent 16
'''queue dropped pkts : 0'''

Class-map (queuing): out-q2 (match-any)
queue-limit percent 1
''' queue dropped pkts : 0'''

Class-map (queuing): out-q3 (match-any)
queue-limit percent 1
''' queue dropped pkts : 0 '''

Class-map (queuing): out-q-default (match-any)
queue-limit percent 82
bandwidth remaining percent 25
''' queue dropped pkts : 0'''

==Second Stage Port QoS Occurs==

For the ingress QoS, we are concerned about the Receive side QoS parameters in the show queuing command.

Use the '''show policy-map''' command to view queue drops .

The commands to troubleshoot Port QoS are

* '''show queuing interface''' ''interface''
* '''show policy-map interface'''

==Layer 2 Source/Destination MAC Processing==

In this step, the ASIC submits the packet headers to theLayer 2 engine for lookup, and the Layer 2 engine performs source/destination MAC processing.

To validate forwarding of the Layer 2 engine, we should first look at the centralized mac table aggregated on the supervisor to validate whether the mac addresses are correlated as we expect them, and assigned to the ports where we expect the Mac’s to reside.

Based off of this, we can then validate the hardware programming on the ingress linecard to validate that our mac address table is properly programmed into the hardware based Layer 2 engine on the linecard.

We first will look at the mac address table, then we can ensure programming is properly occurring in the hardware table.

The commands used to accomplish this are as follows:
* '''show mac address-table'''
* '''show hardware mac address-table''' ''module'' '''interface''' ''interface''

To drill down on a specific MAC address, we can use the grep function with these commands to validate the mac is associated with a particular port, and that the hardware programming reflects that.

* '''show mac address-table | grep''' ''macaddress''
* '''show hardware mac address-table''' ''module'' '''interface''' ''interface'' | '''grep''' ''macaddress''

{{note|When evaluating the Hardware mac table, if the Index is set to 0x00400, or the GM bit is set to “1”, that traffic will be routed. For example, you will see the index set to 0x00400 and GM bit set to 1 for traffic destined to the mac address local to the device}}

PHX2-N7K-1# '''show mac address-table'''
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+------+----------------
G - 0023.ac67.dd41 static - False False sup-eth1(R)
G 5 0023.ac67.dd41 static - False False sup-eth1(R)
* 5 0000.0c07.ac01 dynamic 0 False False Eth1/1
* 5 000c.2943.a67e dynamic 180 False False Eth1/1
* 5 '''000c.294b.c5ca''' dynamic 0 False False '''Eth1/1'''
* 5 000d.ece2.0640 dynamic 180 False False Eth1/1
* 5 0013.5f32.aa80 dynamic 0 False False Eth1/1
* 5 0018.8b45.41b7 dynamic 0 False False Eth1/1
* 5 0019.bb2f.4871 dynamic 0 False False Eth1/1
* 5 0019.bbe5.f3b8 dynamic 1230 False False Eth1/1
* 5 001a.4b33.ccdc dynamic 1080 False False Eth1/1
* 5 001a.4ba8.6a9c dynamic 1680 False False Eth1/1
* 5 001b.210a.87f9 dynamic 600 False False Eth1/1
* 5 001b.d46f.70e0 dynamic 60 False False Eth1/1
* 5 001c.c4e5.ac9a dynamic 150 False False Eth1/1
* 5 0023.ac64.6f7c dynamic 1230 False False Eth1/1
* 5 0024.986d.21c8 dynamic 270 False False Eth1/1

PHX2-N7K-1# '''show hardware mac address-table 1 int eth1/1'''
Valid| PI | BD | MAC | Index | Stat| SW | Modi| Age | Tmr | GM | Sec|
TR | NT | RM | RMA | Cap | Fld | Always | | | |
| ic | | fied| Byte| Sel | | ure| AP | FY | | | TURE| | Learn
-----+----+-------+---------------+--------+-----+----+-----+-----+-----+----+--
--+----+----+----+-----+-----+-----+--------
1 1 2 '''000c.294b.c5ca''' 0x00422 0 3 0 67 1 0
0 0 0 0 0 0 0 0
1 1 2 0050.567e.58e6 0x00422 0 3 0 68 1 0
0 0 0 0 0 0 0 0
1 1 2 0050.56aa.6067 0x00422 0 3 0 67 1 0
0 0 0 0 0 0 0 0
1 1 2 00c0.b72e.cfa0 0x00422 0 3 0 67 1 0
0 0 0 0 0 0 0 0
1 1 2 0018.8b45.41b7 0x00422 0 3 0 68 1 0
0 0 0 0 0 0 0 0
1 1 2 0013.5f32.aa80 0x00422 0 3 0 68 1 0
0 0 0 0 0 0 0 0
1 1 2 0050.56aa.75ca 0x00422 0 3 0 64 1 0
0 0 0 0 0 0 0 0
1 1 2 00a0.9811.a233 0x00422 0 3 0 39 1 0
0 0 0 0 0 0 0 0

PHX2-N7K-1# '''show mac address-table | grep 000c.294b.c5ca'''
* 5 000c.294b.c5ca dynamic 150 False False '''Eth1/1'''

PHX2-N7K-1# '''show hardware mac address-table 1 int eth 1/1 | grep 000c.294b.c5ca'''

1 1 2 '''000c.294b.c5ca 0x00422''' 0 3 0 67 1 0
0 0 0 0 0 0 0 0

==Layer 3 Engine Processing==

After the Layer 2 engine is finished, it sends the header to the Layer 3 engine. The layer 3 engine applies layer 3 intelligent features, to all packets, and layer 3 forwarding, to routed packets. As such, this section will divide the troubleshooting into two components, the layer 3 features applied to all packets, and the layer 3 forwarding.

The layer 3 features which are applied to all packets include the below features :

1) ACL
2) QoS
3) Netflow
4) Hardware IPS

Following the evaluation of the features, we will evaluate the layer 3 forwarding troubleshooting.

===Layer 3 Engine Processes Layer 3 Features===

We’ll drill through each of the layer 3 features below, looking at one feature at a time.

The first feature we will look at is ACL. To troubleshoot ACL, we want to evaluate the configuration, and any relevant hit counters. We then can identify if the hardware on the linecard is programming the ACL.

It is important to note, that if you wish to see per ACL counters, you must enable “statistics per-entry” in the ACL.

The commands to troubleshoot ACL are as follows:

* '''show access-lists''' ''name''
* '''show hardware access-lists module''' ''module''
* '''show hardware access-lists resource-utilization module''' ''module''

PHX2-N7K-1# '''show access-lists sample-86'''

IP access list sample-86
'''statistics per-entry'''
10 permit ospf any any '''[match=0]'''
20 permit pim any any [match=0]
30 permit tcp any any [match=0]
40 permit ip any any [match=0]

PHX2-N7K-1# '''show hardware access-list mod 1'''
VLAN 86 :
=========
No ingress policies
No Netflow profiles in ingress direction

Policies in egress direction:
Policy type Policy Id Policy name
------------------------------------------------------------
RACL 4 '''sample-86'''
No Netflow profiles in egress direction

Tcam 1 resource usage:
----------------------
Label_b = 0x800
Bank 0
------
IPv4 Class
Policies: RACL(sample-86) [Merged]
1 tcam entries

1 l4 protocol cam entries
0 mac etype/proto cam entries
0 lous
0 tcp flags table entries
0 adjacency entries

VDC-1 Ethernet1/1 :
====================
Policies in ingress direction:
Policy type Policy Id Policy name
------------------------------------------------------------
QoS 1

No Netflow profiles in ingress direction

Policies in egress direction:
Policy type Policy Id Policy name
------------------------------------------------------------
QoS 2
No Netflow profiles in egress direction

...

VDC-1 CoPP :
====================
Policies in ingress direction:
Policy type Policy Id Policy name
------------------------------------------------------------
QoS 3

No Netflow profiles in ingress direction

Tcam 1 resource usage:
----------------------
Label_b = 0x1
Bank 1
------
IPv4 Class
Policies: QoS()
'''100 tcam entries '''
IPv6 Class
Policies: QoS()
'''73 tcam entries '''

3 l4 protocol cam entries
0 mac etype/proto cam entries
2 lous
0 tcp flags table entries
0 adjacency entries

No egress policies
No Netflow profiles in egress direction

PHX2-N7K-1# '''show hardware access-list resource utilization mod 1'''

ACL Hardware Resource Utilization (Module 1)
--------------------------------------------
Used Free Percent
Utilization
-----------------------------------------------------
Tcam 0, Bank 0 1 16383 0.00
Tcam 0, Bank 1 2 16382 0.01
Tcam 1, Bank 0 6 16378 0.03
Tcam 1, Bank 1 176 16208 1.07

The next feature we will look at is the QoS troubleshooting for the Nexus. Note, we will have QoS applied, potentially, on both ingress and egress. So we should interrogate both the ingress and egress QoS.

The commands to troubleshoot QoS are

* '''show policy-map interface''' ''interface''

PHX2-N7K-1# '''show policy-map interface eth 1/2'''

Global statistics status : enabled
Ethernet1/2

Service-policy (queuing) '''input:''' default-in-policy
policy statistics status: enabled

Class-map (queuing): in-q1 (match-any)
queue-limit percent 50
bandwidth percent 80
'''queue dropped pkts : 0 '''

Class-map (queuing): in-q-default (match-any)
queue-limit percent 50
bandwidth percent 20
''' queue dropped pkts : 0 '''

'''Service-policy (qos) output: test-police-86 '''
'''policy statistics status: enabled'''

''' Class-map (qos): test-police-86 '''(match-all)
''' Match: dscp 18'''
''' police cir 100 mbps bc 200 ms '''

Service-policy (queuing) '''output: ''' default-out-policy
policy statistics status: enabled

Class-map (queuing): out-pq1 (match-any)
priority level 1
queue-limit percent 16
'''queue dropped pkts : 0 '''

Class-map (queuing): out-q2 (match-any)
queue-limit percent 1
''' queue dropped pkts : 0 '''

Class-map (queuing): out-q3 (match-any)
queue-limit percent 1
''' queue dropped pkts : 0 '''

Class-map (queuing): out-q-default (match-any)
queue-limit percent 82
bandwidth remaining percent 25
queue dropped pkts : 0

Netflow processing also has portions which occur in hardware. For netflow, we collect statistics in hardware on the linecards. We then, can export them via software.

The commands to troubleshoot Netflow are

* '''show flow interface'''
* '''show flow record'''
* '''show flow monitor'''
* '''show hardware flow ip module''' ''module''

PHX2-N7K-1# '''show flow interface vlan 86'''
Interface Vlan86:
Monitor: sample-86
Direction: Input
Monitor: sample-86
Direction: Output

PHX2-N7K-1# '''show flow record'''
Flow record netflow-original:
Description: Traditional IPv4 input NetFlow with origin ASs
No. of users: 2
Template ID: 256
Fields:
match ipv4 source address
match ipv4 destination address
match ip protocol
match ip tos
match transport source-port
match transport destination-port
match interface input
match interface output
match flow direction
collect routing source as
collect routing destination as
collect routing next-hop address ipv4
collect transport tcp flags
collect counter bytes
collect counter packets
collect timestamp sys-uptime first
collect timestamp sys-uptime last

PHX2-N7K-1# '''show flow monitor'''
Flow Monitor Solarwinds1:
Use count: 0
Flow Record: netflow-original
Flow Exporter: Solarwinds
Flow Monitor sample-86:
Use count: 2
Flow Record: netflow-original

switch(config)# '''show hardware flow ip module 8'''

D - Direction; L4 Info - Protocol:Source Port:Destination Port
IF - Interface: ()ethernet, (S)vi, (V)lan, (P)ortchannel, (T)unnel
TCP Flags: Ack, Flush, Push, Reset, Syn, Urgent

D IF SrcAddr DstAddr L4 Info PktCnt TCP Flags
-+-----+---------------+---------------+---------------+----------+-----------
I 8/26 007.002.000.002 007.001.000.002 000:00000:00000 0000421885 . . . . . .
I 8/25 007.001.000.002 007.002.000.002 000:00000:00000 0000421900 . . . . . .
O 8/25 007.002.000.002 007.001.000.002 000:00000:00000 0000422213 . . . . . .
O 8/26 007.001.000.002 007.002.000.002 000:00000:00000 0000422228 . . . . . .

Cisco NX-OS supports a hardware based intrusion detection system that checks for ip packet verification. These checks handle well known, and unusable traffic types which can be witnessed during malicious activity, such as if the source is a broadcast address, or if the destination is the 0.0.0.0 address. You can validate if any of these checks are dropping packets.

{{note|It has been shown in the field, that frequently it is advantageous to disable IP fragment verification. This is done via the below command
* '''no hardware ip verify fragment'''
}}

To validate if you are seeing any drops because of ip packet verification, you can use the below command.

* '''show hardware forwarding ip verify'''
* '''show hardware rate-limit module''' ''module''

PHX2-N7K-1# '''show hardware forwarding ip verify'''

IPv4 and v6 IDS Checks Status Packets Failed
-----------------------------+---------+------------------
address source broadcast Enabled 0
address source multicast Enabled 0
address destination zero Enabled 0
address identical Enabled 0
address source reserved Enabled 8
address class-e Disabled --
checksum Enabled 0
protocol Enabled 0
fragment Enabled 0
length minimum Enabled 0
length consistent Enabled 0
length maximum max-frag Enabled 0
length maximum udp Disabled --
length maximum max-tcp Enabled 0
tcp flags Disabled --
tcp tiny-frag Enabled 0
version Enabled 0
-----------------------------+---------+------------------
IPv6 IDS Checks Status Packets Failed
-----------------------------+---------+------------------
length consistent Enabled 0
length maximum max-frag Enabled 0
length maximum udp Disabled --
length maximum max-tcp Enabled 0
tcp tiny-frag Enabled 0
version Enabled 0

PHX2-N7K-1# '''show hardware rate-limit module 1'''

Units for Config: packets per second
Allowed, Dropped & Total: aggregated since last clear counters

Rate Limiter Class Parameters
------------------------------------------------------------
layer-3 mtu Config : 500
Allowed : 0
'''Dropped : 0 '''
Total : 0

layer-3 ttl Config : 500
Allowed : 0
'''Dropped : 0 '''
Total : 0

layer-3 control Config : 10000
Allowed : 0
''' Dropped : 0 '''
Total : 0

layer-3 glean Config : 100
Allowed : 0
'''Dropped : 0'''
Total : 0

layer-3 multicast directly-connected Config : 3000
Allowed : 0
''' Dropped : 0'''
Total : 0

layer-3 multicast local-groups Config : 3000
Allowed : 0
''' Dropped : 0'''
Total : 0

layer-3 multicast rpf-leak Config : 500
Allowed : 0
''' Dropped : 0 '''
Total : 0

layer-2 storm-control Config : Disabled

access-list-log Config : 100
Allowed : 0
''' Dropped : 0 '''
Total : 0

copy Config : 30000
Allowed : 0
'''Dropped : 0'''
Total : 0

receive Config : 30000
Allowed : 20450903
'''Dropped : 0 '''
Total : 20450903

layer-2 port-security Config : Disabled

layer-2 mcast-snooping Config : 10000
Allowed : 906
'''Dropped : 0 '''
Total : 906

===Layer 3 forwarding for Routed Traffic===

The Layer 3 engine will only perform Layer 3 forwarding for traffic that is routed through the router. This is traffic which has been sent to the MAC address of a valid routed interface, local to the router.

To troubleshoot the routed traffic, we need to perform the following tasks:
1. Ensure that the control plane routing is correct.
2. Ensure that the hardware forwarding entries on the ingress module have the corresponding information.

{{note|All routing of traffic is performed on the forwarding engine of the ingress module.}}

For our example, we will troubleshoot a route 86.86.87.0/24, which is set to a next hop of 86.86.86.1, and set to route out of VLAN 86 (an SVI).

We first will look at the route, ensure it is set to the correct next hop (86.86.86.1), and set to route out of VLAN 86. We will then want to ensure that we have a corresponding ARP entry associated with this next hop, and validate that the adjacency is in the adjacency table.

As we can see below, 86.86.87.0/24 is set to route to 86.86.86.1, out VLAN 86. This next hop is associated with MAC address 0011.aabb.ccdd. We will use this information to investigate the hardware, next.

The commands used to accomplish this are as follows:

* '''show ip route''' (prefix)
* '''show ip arp''' (nexthop)
* '''show ip adjacency'''

PHX2-N7K-1# '''show ip route 86.86.87.0/24'''
IP Route Table for VRF "default"
86.86.87.0/24, 1 ucast next-hops, 0 mcast next-hops
'''*via 86.86.86.1, Vlan86''', [1/0], 01:19:24, static

PHX2-N7K-1# '''show ip arp 86.86.86.1'''

IP ARP Table
Total number of entries: 1
Address Age MAC Address Interface
'''86.86.86.1 - 0011.aabb.ccdd Vlan86 '''

PHX2-N7K-1# '''show ip adjacency '''

IP Adjacency Table for VRF default
Total number of entries: 1
Address Age MAC Address Pref Source Interface
'''86.86.86.1''' 00:00:37 '''0011.aabb.ccdd''' 1 Static ''' Vlan86'''

The above example shows the control plane. Now that we know how things are supposed to work, we can interrogate the hardware to ensure the hardware entries have propagated properly to the Layer 3 hardware engine. We can see that the IP FIB has properly associated 86.86.87.0/24 to the next hop of 86.86.86.1. We can also see, in the hardware entry, that this is routed out VLAN 86, that the RPF is valid if we have enabled RPF Checking), and that the route entry is correctly associated with the MAC address of 0011.aabb.ccdd.

This demonstrates that the routing in the forwarding plane is programmed correctly and that the forwarding will follow the information contained in the routing protocols.

The commands used to accomplish this are as follows:

* '''show ip fib route''' ''prefix'' '''module''' ''module''
* '''show system internal forwarding route''' ''prefix'' ''' detail module''' ''module''

PHX2-N7K-1# '''sho ip fib route 86.86.87.0/24 mod 1'''

IPv4 routes for table default/base

------------------+------------------+---------------------
Prefix | Next-hop | Interface
------------------+------------------+---------------------
'''86.86.87.0/24 86.86.86.1 Vlan86'''

PHX2-N7K-1# '''show system internal forwarding route 86.86.87.1/24 detail mod 1'''

RPF Flags legend:
S - Directly attached route (S_Star)
V - RPF valid
M - SMAC IP check enabled
G - SGT valid
E - RPF External table valid
'''86.86.87.0/24 , Vlan86'''
Dev: 1 , Idx: 0x19001 , RPF Flags: V , DGT: 0 , VPN: 1
'''RPF_Intf_5: Vlan86 ''' (0x55 )
AdjIdx: 0x43005, LIFB: 0 , '''LIF: Vlan86 ''' (0x55 ), DI: 0x0
'''DMAC: 0011.aabb.ccdd''' SMAC: 0023.ac67.dd41

==SFabric Processing Occurs (optional)==

This step occurs if the packet needs to traverse the fabric.

In this step, we need to interrogate if the fabrics are functioning properly, and if their utilization is at an acceptable level.
We can view the fabric status and utilization using the following commands:

* '''show hardware fabric-utilization'''
* '''show module fabric'''

switch(config)# '''show hardware fabric-utilization'''
-----------------------------
Slot Direction Utilization
-----------------------------
2 ingress 3%
2 egress 3%
6 ingress 1%
6 egress 1%

PHX2-N7K-1# '''show module fabric'''
Xbar Ports Module-Type Model Status
--- ----- -------------------------------- ------------------ ------------
1 0 Fabric Module 1 N7K-C7010-FAB-1 ok
2 0 Fabric Module 1 N7K-C7010-FAB-1 ok
3 0 Fabric Module 1 N7K-C7010-FAB-1 ok

Xbar Sw Hw
--- -------------- ------
1 NA 1.0
2 NA 1.0
3 NA 1.0

Xbar MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
1 NA JAF1252AHRB
2 NA JAF1251CABF
3 NA JAF1252AHBL

* this terminal session

==Layer 2 Engine Performs Source/Destination MAC Processing==

To validate forwarding of the Layer 2 engine, we should first look at the centralized MAC table aggregated on the supervisor to validate whether the MAC addresses are correlated as we expect them, and assigned to the ports where we expect the MAC’s to reside.

Based on this, we can then validate the hardware programming on the egress module to validate that our MAC address table is properly programmed into the hardware based Layer 2 engine on the module.

The commands used to accomplish this are as follows:

* '''show mac address-table'''
* '''show hardware mac address-table''' ''module'' '''interface''' ''interface''

The output from these commands are documented in steps 3-4 above.

==Egress Port QoS is Performed==

As the packet nears entry from the Cisco Nexus 7000, one of the final steps is the application of Port QoS. The port level QoS will be able to buffer the traffic in times of congestion. To interrogate the egress QoS, we look at following QoS commands, paying attention to the Transmit QoS:

* '''show queuing interface''' ''interface''
* '''show policy-map interface'''

PHX2-N7K-1# '''show queuing int eth 1/1'''
'''Interface Ethernet1/1 TX Queuing strategy: Weighted Round-Robin'''
'''Port QoS is enabled'''
''' Queuing Mode in TX direction: mode-cos'''
''' Transmit queues [type = 1p7q4t]'''
Queue Id Scheduling Num of thresholds
_____________________________________________________________
1p7q4t-out-q-default WRR 04
1p7q4t-out-q2 WRR 04
1p7q4t-out-q3 WRR 04
1p7q4t-out-q4 WRR 04
1p7q4t-out-q5 WRR 04
1p7q4t-out-q6 WRR 04
1p7q4t-out-q7 WRR 04
1p7q4t-out-pq1 Priority 04

Configured WRR
WRR bandwidth ratios: 25[1p7q4t-out-q-default] 15[1p7q4t-out-q2] 12[1p7q4t-out-q3] 12[1p7q4t-out-q4] 12[1p7q4t-out-q5] 12[1p7q4t-out-q6] 12[1p7q4t-out-q7]
WRR configuration read from HW
WRR bandwidth ratios: 25[1p7q4t-out-q-default] 15[1p7q4t-out-q2] 11[1p7q4t-out-q3] 11[1p7q4t-out-q4] 11[1p7q4t-out-q5] 11[1p7q4t-out-q6] 11[1p7q4t-out-q7]

Configured queue-limit ratios
queue-limit ratios: 78[1p7q4t-out-q-default] 1[1p7q4t-out-q2] 1[1p7q4t-out-q3]
*1[1p7q4t-out-q4] *1[1p7q4t-out-q5] *1[1p7q4t-out-q6] *1[1p7q4t-out-q7]
16[1p7q4t-out-pq1]
* means unused queue with mandatory minimum queue-limit

queue-limit ratios configuration read from HW
queue-limit ratios: 78[1p7q4t-out-q-default] 1[1p7q4t-out-q2] 1[1p7q4t-out-q3]
*1[1p7q4t-out-q4] *1[1p7q4t-out-q5] *1[1p7q4t-out-q6] *1[1p7q4t-out-q7]
16[1p7q4t-out-pq1]
* means unused queue with mandatory minimum queue-limit

Thresholds:
COS Queue Threshold Type Min Max
__________________________________________________________________
0 1p7q4t-out-q-default DT 100 100
1 1p7q4t-out-q-default DT 100 100
2 1p7q4t-out-q-default DT 100 100
3 1p7q4t-out-q-default DT 100 100
4 1p7q4t-out-q-default DT 100 100
5 1p7q4t-out-pq1 DT 100 100
6 1p7q4t-out-pq1 DT 100 100
7 1p7q4t-out-pq1 DT 100 100

...

PHX2-N7K-1# '''show policy-map interface eth 1/2'''
Global statistics status : enabled
Ethernet1/2
Service-policy (queuing) input: default-in-policy
policy statistics status: enabled

Class-map (queuing): in-q1 (match-any)
queue-limit percent 50
bandwidth percent 80
'''queue dropped pkts : 0 '''

Class-map (queuing): in-q-default (match-any)
queue-limit percent 50
bandwidth percent 20
''' queue dropped pkts : 0 '''

Service-policy (queuing) output: default-out-policy
policy statistics status: enabled

Class-map (queuing): out-pq1 (match-any)
priority level 1
queue-limit percent 16
''' queue dropped pkts : 0'''

Class-map (queuing): out-q2 (match-any)
queue-limit percent 1
''' queue dropped pkts : 0 '''

Class-map (queuing): out-q3 (match-any)
queue-limit percent 1
''' queue dropped pkts : 0 '''

Class-map (queuing): out-q-default (match-any)
queue-limit percent 82
bandwidth remaining percent 25
''' queue dropped pkts : 0 '''

==Linksec Encryption Occurs==

The command used to troubleshoot Linksec encryption is

* '''show cts interface''' {'''all''' | ''interface''}

The output from this command is documented in step 2 above.

==Packet is Transmitted==

The final step in the process is the transmission of the frame out of the physical egress port. Troubleshooting of the physical port, is the same as in step 1, and includes the following commands:

* '''show interface''' ''interface''
* '''show interface''' ''interface'' '''transceiver'''

The output from these commands are documented in step 1 above.

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Multicast Traffic

2011-11-30T20:06:35Z

Sukumars: Created page with "Troubleshooting L2/L3 multicast is covered in detail at Cisco-Live presentation. Sections of this presentation covers, both platform independent, and platform specific step by..."

Troubleshooting L2/L3 multicast is covered in detail at Cisco-Live presentation. Sections of this presentation covers, both
platform independent, and platform specific step by step troubleshooting for multicast, among other things. Access to this
presentation is available FREE. Follow the below instructions to access the presentation

1. Visit https://www.ciscolivevirtual.com/

2. Register for free.

3. Click on "Cisco Live Virtual" link.

4. Click on the ‘Sessions’ Tab on top, and select ‘2011 Sessions Catalog’.

5. In the search box, type “BRKCRS-3144” and Submit search.

6. Select the session. You can either View the Session (or) download the presentation in pdf format.

7. Multicast troubleshooting is covered from slides 95 through 112.

Cisco Nexus 7000 Series NX-OS Troubleshooting Guide

2011-11-30T19:56:12Z

Sukumars: /* Organization */

{{Template:Required Metadata}}
This article provides guidance for troubleshooting issues that may appear when using Cisco Nexus 7000 Series. This article introduces tools and methodologies to recognize a problem, determine its cause, and find possible solutions.
{| align="center" border="1"
|align="left"|Welcome to '''Cisco DocWiki'''. We encourage [http://tools.cisco.com/RPF/register/register.do registered Cisco.com users] to contribute to this wiki to improve Cisco product documentation. Note that you cannot log in to DocWiki with Cisco.com "guest" account credentials. 
See [[DocWiki:Terms_of_use|Terms of Use]] and [[DocWiki:About|About DocWiki]] for more information about Cisco DocWiki. 
Select the "edit" tab to edit an article or select the "discussion" tab to submit questions or comments about the article. 

Click [http://www.cisco.com/en/US/products/ps9402/prod_troubleshooting_guides_list.html here] to return to the Cisco Nexus 7000 Series documentation on www.cisco.com.
|}

__TOC__

==Audience and Generating a PDF of This Guide==

This article is for experienced network administrators who configure and maintain NX-OS devices.

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Book PDF]]

==Organization==

This article is organized into the following sections:

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Overview|Troubleshooting Overview]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Installs, Upgrades, and Reboots|Troubleshooting Installs, Upgrades, and Reboots]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Licensing|Troubleshooting Licensing]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VDCs|Troubleshooting VDCs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting CFS|Troubleshooting CFS]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Ports|Troubleshooting Ports]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs|Troubleshooting vPCs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting VLANs|Troubleshooting VLANs]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting STP|Troubleshooting STP]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Routing|Troubleshooting Routing]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Unicast Traffic|Troubleshooting Unicast Traffic]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Multicast Traffic|Troubleshooting Multicast Traffic]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting WCCP|Troubleshooting WCCP]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Memory|Troubleshooting Memory]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Packet Flow Issues|Troubleshooting Packet Flow Issues]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Before Contacting Technical Support|Before Contacting Technical Support]]

[[Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting Tools and Methodology|Troubleshooting Tools and Methodology]]

[[Category:Nexus 7000 Series Switches]]

[[Category:Troubleshooting]]