Cisco MDS SanOS Troubleshooting Guide -- Troubleshooting SANTap
From DocWiki
(→Common Mismatch Problems) |
(→Common Mismatch Problems) |
||
| Line 513: | Line 513: | ||
* Too many ITLs per DPP | * Too many ITLs per DPP | ||
| | | | ||
| - | When replication is enabled, AVT LUNs are created, and | + | When replication is enabled, AVT LUNs are created, and which can increase the ITL count over the limit. (See the "Limitations" section.) |
If Reservation Support is not enabled on the RPA: | If Reservation Support is not enabled on the RPA: | ||
| Line 521: | Line 521: | ||
* The appliance completes recovery of these LUNs and then deletes them before creating more. | * The appliance completes recovery of these LUNs and then deletes them before creating more. | ||
| - | * | + | * The ITL count is not significantly increased. |
If Reservation Support is enabled on the RPA: | If Reservation Support is enabled on the RPA: | ||
Revision as of 10:53, 28 September 2010
This section introduces the basic concepts, methodology, and general troubleshooting guidelines for problems that may occur when configuring SANTap and using the Cisco MDS 9000 Family of multilayer directors and fabric switches.
Contents |
Troubleshooting SANTap
This section describes how to identify and resolve problems that might occur when implementing SANTap. This section includes the following sections:
- Overview
- Definitions
- Limitations
- Initial Troubleshooting Checklist
- SANTap Issues
Overview
 | Note: | SANTap is not supported in Release 3.3.1. |
SANTap is an Intelligent Storage Services feature supported on the Storage Services Module (SSM). The SSM supports SANTap in Cisco MDS SAN-OS Release 2.0(2b) and later. The SANTap feature allows third-party data storage applications, such as long distance replication and continuous backup, to be integrated into the SAN.
SANTap has a control path and a data path. The control path services requests that create and manipulate replication sessions which are sent by an appliance. The control path is implemented using a SCSI-based protocol. An appliance sends requests to a control virtual target (CVT) that the SANTap process creates and monitors. Responses are sent to the control LUN on the appliance. SANTap also allows LUN mapping to appliance virtual targets (AVTs). You can have a maximum of 512 target LUNs.
When introducing SANTap-based applications, SANTap does not require reconfiguration of either the host or the target, and neither the host initiator nor the target is required to be directly connected to an SSM. This operation is accomplished by assigning Cisco-specific WWNs to the virtual initiators (VIs) and data virtual targets (DVTs).
A host initiator or a target can be connected directly to an SSM. However, you must partition the SAN using VSANs. You must configure the host initiator and the DVT in one VSAN and configure the virtual initiator (VI) and the target in another VSAN.
Definitions
Table 23-1 includes brief definitions of some of the common SANTap acronyms and terms.
|
Acronym / Term
|
Definition
| |||
|---|---|---|---|---|
|
AVT |
Appliance virtual target. The portal through which an appliance can complete its synchronization with the target LUN. AVT can be thought of as a host proxy for the appliance. | |||
|
CVT |
Control virtual target. The portal through which an appliance communicates with SANTap. An initiator port on the appliance sends out SANTap Control Protocol requests to the SANTap process. When the request is processed, the response is sent back by the Cisco VI (virtual initiator) to a target port on the appliance. | |||
|
DVT |
Data virtual target. A DVT is created for every port on a multi-ported target that is included in SANTap-based services. The DVT is created in the host VSAN. Once a DVT is created and a host logs into the DVT, SANTap installs a DVTLUN for every configured LUN on the target for this host. | |||
|
ITL |
Initiator/target/LUN tuple. Uniquely identifies a LUN on a target. | |||
|
Session |
A record/object that is created for every ITL whose WRITE I/Os the appliance is interested in. A session can be thought of as a target LUN that requires SANTap-based services. | |||
|
VI |
Virtual initiator. SANTap creates 9 VIs in the appliance and target VSANs. In the appliance, VIs are used to send responses to SANTap CP requests and also to send copies of WRITE I/Os. In the target, VIs are used when the appliance is down and one of the SANTap recovery tools (ARL, PWL-BPR) is enabled.
|
Limitations
This section describes SANTap limitations. Table 23-2 describes the component limitations by SAN-OS and SSI releases.
- One Recover Point Appliance (RPA) cluster can have only one target VSAN.
- SANTap does not support IVR. You cannot run SANTap and IVR together.
|
Attribute
|
Limitation
|
Cisco SAN-OS and SSI Release
|
|---|---|---|
|
Hosts per DVT |
16 |
Prior to Cisco SAN-OS 3.2 |
|
32 |
Cisco SAN-OS 3.2 | |
|
LUNs per host |
256 |
All releases |
|
LUNs per DVT |
1500 |
Cisco SAN-OS 3.2 |
|
1024 |
Prior to Cisco SAN-OS 3.2 | |
|
DVTs per SSM |
32 |
Cisco SAN-OS 3.1(2) with SSI 3.1(2m) and later |
|
16 |
Prior to Cisco SAN-OS 3.1(2) with SSI 3.1(2m) | |
|
Sessions per SSM |
1024 |
Cisco SAN-OS 3.0(2) with SSI 3.0(2j) |
|
2048 |
Cisco SAN-OS 3.1(2b) with SSI 3.1(2m) and later | |
|
LUN ID address |
32-bits |
Cisco SAN-OS 3.2 |
|
16-bits |
Prior to Cisco SAN-OS 3.2 | |
|
DVT LUNs per SSM |
4080 |
All releases |
|
ITLs per DPP |
1500 |
Cisco SAN-OS 3.2 |
|
1024 |
Prior to Cisco SAN-OS 3.2 | |
|
ITLs per SSM |
1024 |
Cisco SAN-OS 3.0(2) with SSI 3.0(2j) |
|
2048 |
Cisco SAN-OS 3.1(2b) with SSI 3.1(2m) | |
|
4080 |
Cisco SAN-OS 3.1(3) with SSI 3.1(3) and later |
Interface Restrictions
When enabled on a per-port basis, SANTap must be enabled on a group of four ports on an SSM. The following restrictions apply:
- The fewest number of interfaces that can be SANTap enabled is four.
- The first interfaces in a group must be 1, 5, 9, 13, 17, 21, 25, or 29. (You can specify fc5 through fc8, but not fc7 through fc10.)
- The groups of four interfaces do not need to be consecutive. For example, you can specify fc1 through fc8 and fc17 through fc20.
Initial Troubleshooting Checklist
Troubleshooting a SANTap problem involves gathering information about the configuration and connectivity of the various SANTap components. Begin your troubleshooting activity as follows:
|
Checklist
|
Check off
|
|---|---|
|
Verify licensing requirements. See Cisco MDS 9000 Family Fabric Manager Configuration Guide. | |
|
Verify that SANTap is enabled on the SSM module of the selected switch. | |
|
Verify the VSAN configuration and zones for the appliance, using the configuration and verification tools for the specific appliance. | |
|
Verify the physical connectivity for any problem ports or VSANs. |
Common Troubleshooting Tools in Fabric Manager
Use the following Fabric Manager procedures to verify the VSAN configuration for the SANTap components.
- Choose Fabricxx > VSANxx to view the VSAN configuration in the Information pane.
- Choose Fabricxx > VSANxx and select the Host or Storage tab in the Information pane to view the VSAN members.
Common Troubleshooting Commands in the CLI
Use the show santap module command to display information about SANTap.
Example 23-1 Display SANTap CVT Information
switch# show santap module 2 cvt
CVT Information :
cvt pwwn = 23:4f:00:0d:ec:09:3c:02
cvt nwwn = 23:9d:00:0d:ec:09:3c:02
cvt id = 135895180
cvt xmap_id = 135895212
cvt vsan = 8
cvt name =
Example 23-2 Display SANTap DVT Information
switch# show santap module 2 dvt
DVT Information :
dvt pwwn = 50:06:0e:80:03:81:32:36
dvt nwwn = 50:06:0e:80:03:81:32:36
dvt id = 136773180
dvt mode = 3
dvt vsan = 12
dvt if_index = 0x1080000
dvt fp_port = 1
dvt name = MYDVT
dvt tgt-vsan = 9
dvt io timeout = 10 secs
dvt lun size handling = 0
dvt app iofail behaviour = 1
dvt quiesce behavior = 1
dvt tgt iofail behavior = 0
dvt appio failover time = 50 secs
dvt inq data behavior = 0
Example 23-3 Display SANTap DVT LUN Information
switch# show santap module 2 dvtlun
DVT LUN Information :
dvt pwwn = 22:00:00:20:37:88:20:ef
dvt lun = 0x0
xmap id = 8
dvt id = 3
dvt mode = 0
dvt vsan = 3
tgt pwwn = 22:00:00:20:37:88:20:ef
tgt lun = 0x0
tgt vsan = 1
Example 23-4 Display SANTap Session Information
switch# show santap module 2 session
Session Information :
session id = 1
host pwwn = 21:00:00:e0:8b:12:8b:7a
dvt pwwn = 50:06:0e:80:03:81:32:36
dvt lun = 0x0
tgt pwwn = 50:06:0e:80:03:81:32:36
tgt lun = 0x0
adt pwwn = 33:33:33:33:33:33:33:00
adt lun = 0x0
aci pwwn = 22:22:22:22:22:22:22:22
cvt pwwn = 23:4f:00:0d:ec:09:3c:02
num ranges = 0
session state = 5
redirect mode = 0
mrl requested 1
MRL : vsan 8 RegionSize 4806720, DiskPWWN 0x234f000dec093c02, DiskLun 0x 1, startLBA 1
pwl requested 1
PWL : type 2, UpdatePol 2, RetirePolicy 4, pwl_start 1
iol requested 0
Example 23-5 Display SANTap AVT Information
switch# show santap module 2 avt
AVT Information :
avt pwwn = 2a:4b:00:05:30:00:22:25
avt nwwn = 2a:60:00:05:30:00:22:25
avt id = 12
avt vsan = 4
avt if_index = 0x1080000
hi pwwn = 21:00:00:e0:8b:07:61:aa
tgt pwwn = 22:00:00:20:37:88:20:ef
tgt vsan = 1
Example 23-6 Display SANTap AVT LUN Information
switch# show santap module 2 avtlun
AVT LUN Information :
avt pwwn = 2a:4b:00:05:30:00:22:25
avt lun = 0x0
xmap id = 16
avt id = 12
tgt lun = 0x0
Example 23-7 Display SANTap Remote Virtual Terminal Information
switch# show santap module 2 rvt
RVT Information :
rvt pwwn = 2a:61:00:05:30:00:22:25
rvt nwwn = 2a:62:00:05:30:00:22:25
rvt id = 17
rvt vsan = 4
rvt if_index = 0x1080000
Example 23-8 Display SANTap Remote Virtual Terminal LUN Information
switch# show santap module 2 rvtlun
RVT LUN Information :
rvt pwwn = 2a:61:00:05:30:00:22:25
rvt lun = 0x0
xmap id = 22
rvt id = 17
app pwwn = 22:00:00:20:37:39:b1:00
app lun = 0x0
app vsan = 1
Use the following commands to display more advanced troubleshooting information for SANTap.
- show tech-support
- show santap module 2 tech-support
- show isapi tech-support
- show santap vttbl dvt dvt-pwwn
Messages, Logs, and Databases
The following log files and databases can provide helpful information when troubleshooting SANTap problems:
- Look for SSM_CRIT in Sup syslog messages.
- SANTap logs are available with the show isapi tech-support CLI command. Search for the strings "Error" and "failed" or "failure."
- Review the FCNS and FLOGI databases using the show fcns and show flogi CLI commands.
SANTap Issues
This section includes the following topics:
- Host Login Problems
- ITL Problems
- Common Mismatch Problems
Host Login Problems
Host login problems can be caused by DVT limitations, connectivity, or other issues.
To diagnose host login problems, follow these steps:
1. Use the show santap vttbl dvt dvt-pwwn command to display SANTap information for the DVT.
2. Use the show santap vttbl dvt dvt-pwwn host-pwwn command to display SANTap information for the DVT and the host.
3. Enable logging on the appropriate SSM module.
4. Enter the following debug commands:
- a. debug vsd vfc-felogin
- b. debug partner 0x92810000 d1
- c. debug partner 0x92810000 d2
5. Review the logs to determine the problem, and then take the appropriate corrective action.
ITL Problems
An ITL problem may occur if the number of ITLs exceeds the limitations for the version of Cisco SAN-OS and SSI in use. For specific ITL limitations, see the "Limitations" section.
To diagnose and resolve ITL problems, follow these steps:
1. Use the show isapi dpp 4 queue command to display DPP queue information.
2. Verify that the number of ITLs on a DPP is within the limitations for the version of Cisco SAN-OS and SSI in use. Use the show isapi dpp 4 queue incl LUN and show isapi dpp 4 queue count commands.
3. Verify that the number of ITLs on an SSM is within the limitations for the version of Cisco SAN-OS and SSI in use. Enter the show isapi dpp all queue command, using the incl LUN and count parameters.
Common Mismatch Problems
Problems are often caused by mismatching component versions, or using devices that are not supported by the interoperability matrix.
Table 23-3 lists common mismatch situations:
|
Problem
|
Description
|
|---|---|
|
Version mismatch between SSM and the RPA. |
The version of SSM and the version of the replication appliance are not compatible. |
|
Version mismatch between the Supervisor and the SSI image. |
The version of the Supervisor and the SSI image are not compatible. |
|
Hosts, targets, HBAs or switches are not supported by the interoperability matrix. | |
|
CVT is in the host VSAN. |
The CVT is the portal through which the appliance communicates with SANTap, and cannot be in the host VSAN. |
|
IVR and SANTap are being used together. |
IVR and SANTap both perform straddling across VSANs and cannot be used together. |
|
VIs in a DVT VSAN (CVT and DVT in the same VSAN). |
This results in the creation of one CPP VI and eight DPP VIs in the DVT VSAN. The VIs attempt to login to DVT, resulting in non-deterministic behavior. |
|
Overprovisioning, including:
|
When replication is enabled, AVT LUNs are created, and which can increase the ITL count over the limit. (See the "Limitations" section.) If Reservation Support is not enabled on the RPA:
If Reservation Support is enabled on the RPA:
|
|
Same pWWN occurs more than once in the same VSAN. |
Misconfiguration can result in two DVTs (or two VIs) with the same WWN in the same VSAN. For example, assume that two DVTs are created on different SSMs or on different DPPs. Both of these DVTs have the same back-end VSAN. When a host HBA logs into both DVTs, an attempt is made to create two VIs with the same WWN in the same back-end VSAN. This results in non-deterministic behavior. |
|
A host is moved from the front-end VSAN to the back-end VSAN. |
There is a VI in the back-end VSAN with the same pWWN as the host. Before you can move the host:
The host can now be safely moved. |
|
Inaccurate zoning. |
Zoning solutions differ, based on the Cisco SAN-OS and SSI versions in use.
|
|
Adding and removing hosts without performing a purge. |
If you have 16 hosts and you remove one and add another, the new host will not see the LUNs. In this situation, perform a purge to clear one of the 16 entries after removing the host. Then you can add the new host to the DVT. |
