Nexus 5000 with Brocade - FcoE/FC
This document describes Data Center technologies(FC and FCoE) and provides a straight forward simple configuration to a N5k connected to a Brocade48000; in order to get FCoE operation.
This Design assumes a Nexus 5020 unit, with a Brocade switch 48000
Data Center Switching:
The Cisco Nexus family of switches is a primary part of the unified fabric pillar of the Cisco Data Center Business Advantage architectural framework. These switches are designed to meet the stringent requirements of the next-generation data center. Not simply bigger or faster, these switches offer the following advantages:
* Infrastructure that can be scaled cost-effectively and that helps you increase energy, budget, and resource efficiency * Transport 10 Gigabit Ethernet and unified fabric and can handle virtualization, Web 2.0 applications, and cloud computing * Operational continuity where system availability is assumed and maintenance windows are rare or nonexistent
The Cisco Nexus 5000 Series Switches help you transform the data center with innovative, standards-based, multilayer, multiprotocol, and multipurpose Ethernet-based fabric. Now you can help enable any transport over Ethernet, including Layer 2 and Layer 3 traffic, and storage traffic, all on one common data center-class platform.
FCoE: FCoE is Fiber channel over ethernet.
Fiber channel (FC): Fiber channel allows for higher bandwidth transport (10Gig). Fibre Channel, or FC, is a gigabit-speed network technology primarily used for storage networking. Fibre Channel is standardized in the T11 Technical Committee of the InterNational Committee for Information Technology Standards (INCITS), an American National Standards Institute (ANSI). Fibre Channel was primarily used in the supercomputer field, but now, has become the standard connection type for storage area networks (SAN) in enterprise storage. Despite its name, Fibre Channel signalling can run on both twisted pair copper wire and fiber-optic cables. Cisco Nexus 5000 Series switches support Fibre Channel over Ethernet (FCoE), which allows Fibre Channel and Ethernet traffic to be carried on the same physical Ethernet connection between the switch and the servers. The Fibre Channel portion of FCoE is configured as a virtual Fibre Channel interface. Logical Fibre Channel features (such as interface mode) can be configured on virtual Fibre Channel interfaces.
Each physical Fibre Channel interface in a switch may operate in one of several port modes: E mode, TE mode, F mode, and SD mode. By default, Cisco Nexus 5000 Series switches operate in fabric mode. In this mode, the switch provides standard Fibre Channel switching capability and features. In fabric mode, each switch that joins a SAN is assigned a domain ID. Each SAN (or VSAN) supports a maximum of 239 domain IDs, so the SAN has a limit of 239 switches. In a SAN topology with a large number of edge switches, the SAN may need to grow beyond this limit. NPV alleviates the domain ID limit by sharing the domain ID of the core switch among multiple edge switches. in NPV mode, the edge switch relays all traffic from server-side ports to the core switch. The core switch provides F port functionality (such as login and port security) and all the Fibre Channel switching capabilities. The edge switch appears as a Fibre Channel host to the core switch and as a regular Fibre Channel switch to its connected devices.
In NPV mode, the edge switch (Nexus) relays all traffic to the core switch (Brocade), which provides the Fibre Channel switching capabilities. The edge switch shares the domain ID of the core switch. To convert a switch into NPV mode, you set the NPV feature to enabled. This configuration command automatically triggers a switch reboot. You cannot configure NPV mode on a per-interface basis. NPV mode applies to the entire switch. All interfaces from the edge switch (Nexus) to the core switch (Brocade) are configured as proxy N ports (NP ports). An NP uplink is a connection from an NP port on the edge switch to an F port on the core switch. When an NP uplink is established, the edge switch sends a fabric login message (FLOGI) to the core switch, and then (if the FLOGI is successful) it registers itself with the name server on the core switch. Subsequent FLOGIs from end devices connected to this NP uplink are converted to fabric discovery messages (FDISCs). (show flogi database and show fcns database)
Best Practice Design Objectives
The Goal of this document is to Get N5K connected to a Brocade48000; in order to get FCoE operation
Best Practice Design Technology Considerations
When configuring virtual interfaces, note the following guidelines and limitations:
•Each virtual Fibre Channel interface must be bound to an FCoE-enabled Ethernet interface. FCoE is supported on 10-Gigabit Ethernet interfaces. •Each virtual Fibre Channel interface is associated with only one VSAN. •Any VSAN with associated virtual Fibre Channel interfaces must be mapped to a dedicated FCOE-enabled VLAN. •FCoE is not supported on private VLANs. •Enable NPV feature
Reference Design Example
!configure FCoE enabled vlan to assosiate with the vsan (vsan to vlan mapping) !
vlan 30 fcoe vsan 30 name FCoE
! !configure the layer 2 server vlan !
vlan 1200 name servers_test
! !configure the trunks connecting to the storage system (ethernet trunk) !
interface Ethernet1/30 description HP Chassis C7000 switchport mode trunk switchport trunk native vlan 1200 switchport trunk allowed vlan 30,1200 spanning-tree port type edge trunk
interface Ethernet1/32 description HP Chassis C7000-NIC2 switchport mode trunk switchport access vlan 1200 switchport trunk native vlan 1200 switchport trunk allowed vlan 30,1200 spanning-tree port type edge trunk
! !configure the virtual fiber channel --The FC portion of FCoE is configured as a vfc interface. !
interface vfc30 bind interface Ethernet1/30 no shutdown
interface vfc32 bind interface Ethernet1/32 no shutdown
! !create the vsan, give it a name and associate ports with it (vsan to vlan mapping) !
vsan database vsan 30 name "FCoE"
vsan database vsan 30 interface vfc30 vsan 30 interface vfc32 vsan 30 interface fc2/1 vsan 30 interface fc2/2
interface fc2/1 switchport mode NP switchport description Brocade_48000 no shutdown
interface fc2/2 switchport mode F switchport description C7000 no shutdown
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/npv.html#wp1508306 http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli_rel_4_0_1a/FCintf.html http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli_rel_4_0_1a/adv.html#wp1296191 http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli_rel_4_0_1a/fcoe.html#wpxref22966 http://www.cisco.com/en/US/docs/storage/san_switches/mds9000/interoperability/guide/intopgd.html https://supportforums.cisco.com/message/474214 http://www.networkworld.com/community/node/42105