Cisco MDS SanOS Troubleshooting Guide -- Before Contacting Technical Support

From DocWiki

Latest revision as of 09:16, 8 November 2010

This section introduces the basic concepts, methodology, and general troubleshooting guidelines for problems that may occur when configuring and using the Cisco MDS 9000 Family of multilayer directors and fabric switches. This section discusses the steps to be taken before contacting TAC.

Before Contacting Technical Support

This appendix describes the steps to perform before calling for technical support for any Cisco MDS 9000 Family multilayer director and fabric switch. This appendix includes the following sections:

• Steps to Perform Before Calling TAC
• Using Core Dumps

Note:

If you purchased Cisco support through a Cisco reseller, contact the reseller directly. If you purchased support directly from Cisco, contact Cisco Technical Support at this URL: [http://www.cisco.com/warp/public/687/Directory/DirTAC.shtml http://www.cisco.com/warp/public/687/Directory/DirTAC.shtm

Steps to Perform Before Calling TAC

At some point, you may need to contact your customer support representative or Cisco TAC for some additional assistance. This section outlines the steps that you should perform prior to contacting your next level of support, because you can reduce the amount of time spent resolving the issue.

Note:

Do not reload the module or the switch at least until you have completed Step 1. Some logs and counters are kept in volatile storage and will not survive a reload.

To prepare for contacting your customer support representative, follow these steps:

1. Collect switch information and configuration. This should be done before and after the issue has been resolved. The following three methods each provide the same information:

a. Select Tools > Show Tech Support in Fabric Manager. Fabric Manager can capture switch configuration information from multiple switches simultaneously. The file can be saved on the local PC.

b. Configure your Telnet or SSH application to log the screen output to a text file. Use the terminal length 0 CLI command and then use the show tech-support details CLI command.

c. Use the tac-pac filename; CLI command to redirect the output of the show tech-support details CLI command to a file, and then gzip the file.

 <nowiki>switch# tac-pac bootflash://showtech.switch1 <nowiki>

If no filename is specified, the file is created as volatile:show_tech_out.gz. The file should then be copied from the switch using the procedure outlined in the "Copying Files to or from the Switch" section.

2. If an error occurs in Fabric Manager, take a screen shot of the error. In Windows, press Alt PrintScreen to capture the active window, or press only PrintScreen to capture the entire desktop. Then paste this into a new Microsoft Paint (or similar program) session and save the file.

3. Capture the exact error codes you see in the message logs from either Fabric Manager or the CLI.

a. Click the Logs tab in the Map pane in Fabric Manager or choose Switches > Events to see the recent list of messages generated.

b. Copy the error from the message log, which can be displayed using either the show logging log CLI command or the show logging last number command to view the last lines of the log.

4. Answer the following questions before calling for technical support:

• On which switch, host bus adapter (HBA), or storage port is the problem occurring?
• Which Cisco SAN-OS software, driver versions, operating systems versions and storage device firmware are in your fabric?
• What is the network topology? (In Fabric Manager, go to Tools > Show Tech Support and check the Save Map check box.)
• Were any changes being made to the environment (zoning, adding modules, upgrades) prior to or at the time of this event?
• Are there other similarly configured devices that could have this problem, but do not?
• Where was this problematic device connected (which MDS switch and interface)?
• When did this problem first occur?
• When did this problem last occur?
• How often does this problem occur?
• How many devices have this problem?
• Were any traces or debug output captured during the problem time? What troubleshooting steps have you attempted? Which, if any, of the following tools were used?
  • FC Analyzer, PAA-2, Ethereal, local or remote SPAN
  • CLI debug commands
  • FC traceroute, FC ping
  • Fabric Manager or Device Manager tools

5. Is your problem related to a software upgrade attempt?

• What was the original Cisco SAN-OS version?
• What is the new Cisco SAN-OS version?
• Did you use Fabric Manager or the CLI to attempt this upgrade?
• Collect the output from the following commands:
  • show install all status
  • show system internal log install
  • show system internal log install details
  • show log nvram
• Forward them to your customer support representative.

6. If your problem is related to zoning, use the show zone tech-support CLI command to gather relevant information.

Copying Files to or from the Switch

It may be required to move files to or from the switch. These files may include log, configuration, or firmware files.

Copying Files Using Device Manager

To copy the configuration from the switch using Device Manager, follow these steps:

1. Choose Admin > Copy Configuration. You see the Copy Configuration dialog box.

2. Set the To field to the server where you want to copy the configuration file to.

3. Set the From field to running or startup configuration.

4. Select the protocol you want to use to copy the file from the switch.

4. Select Apply to copy the file.

To copy files to the switch using Device Manager, follow these steps:

1. Choose Admin > Flash Files. You see the list of files in the chosen device and partition.

2. Select Copy to copy a file. You see the copy file dialog box.

3. Select the protocol that you want to use to copy the file to the switch.

4. Set the server address and the file that you want to copy.

5. Select Apply to copy the file.

Copying Files Using the CLI

The CLI offers a broad range of protocols to use for copying to or from the switch. Note that the switch always acts as a client, such that an ftp/scp/tftp session will always originate from the switch and either push files to an external system or pull files from an external system.

 File Server: 172.22.36.10

 File to be copied to the switch: /etc/hosts

The copy CLI command supports four transfer protocols and 12 different sources for files.

ca-9506# copy ?
 bootflash: Select source filesystem
 core: Select source filesystem
 debug: Select source filesystem
 ftp: Select source filesystem
 licenses Backup license files
 log: Select source filesystem
 modflash: Select source filesystem
 nvram: Select source filesystem
 running-config Copy running configuration to destination
 scp: Select source filesystem
 sftp: Select source filesystem
 slot0: Select source filesystem
 startup-config Copy startup configuration to destination
 system: Select source filesystem
 tftp: Select source filesystem
 volatile: Select source filesystem

Use the following syntax to use secure copy (scp) as the transfer mechanism:

 "scp:[//[username@]server][/path]"

To copy /etc/hosts from 172.22.36.10 using the user user1, where the destination would be hosts.txt, use the following command:

switch# copy scp://user1@172.22.36.10/etc/hosts bootflash:hosts.txt
 user1@172.22.36.10's password:
 hosts 100% |*****************************| 2035 00:00

To back up the startup-configuration to a SFTP server, use the following command:

switch# copy startup-config sftp://user1@172.22.36.10/MDS/startup-configuration.bak1
 
 Connecting to 172.22.36.10...
 User1@172.22.36.10's password:

switch#

Tip
Backing up the startup-configuration to a server should be done on a daily basis and prior to any changes. A short script could be written to be run on the MDS to perform a save, and then a backup of the configuration. The script only needs to contain two commands: copy running-configuration startup-configuration and then copy startup-configuration tftp://server/name. To execute the script use: run-script filename.

Using Core Dumps

Core dumps are available in situations where unknown problems exist. Dumps are sent to a TFTP server or to a flash card in slot0: of the local switch. You should set up your switch to generate core dumps under the instruction of your customer support representative. Core dumps are decoded by technical support engineers.

Best practice is to set up cores dumps to go to a TFTP server. These core dumps can be e-mailed directly to your customer support representative.

Setting Up Core Dumps Using the CLI

Use the system cores CLI command to set up core dumps on your switch.

switch# system cores tftp://10.91.51.200/jsmith_cores
switch# show system cores
Cores are transferred to tftp://10.91.51.200/jsmith_cores

Note:

The filename (indicated by jsmith_cores) must exist in the TFTP server directory.