Nexus 7000 - OTV - Design and Configuration Example
From DocWiki
(→Design & Prerequisites) |
(→Procedures) |
||
| (6 intermediate revisions not shown) | |||
| Line 12: | Line 12: | ||
| 30 min (assuming all pre-requisites are met) | | 30 min (assuming all pre-requisites are met) | ||
|} | |} | ||
| + | ===Overview=== | ||
| + | For additional information regarding Overlay Transport Virtualization (OTV) on Cisco NX-OS devices, refer to the overview chapter available in the ''Cisco Nexus 7000 Series NX-OS OTV Configuration Guide, Release 5.x'' available at: | ||
| + | |||
| + | http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/otv/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_OTV_Configuration_Guide__Release_5.x_chapter1.html | ||
| + | |||
===Diagram=== | ===Diagram=== | ||
| Line 36: | Line 41: | ||
## Each site and provider / WAN network. | ## Each site and provider / WAN network. | ||
# Ensure that there is L3 connectivity between all the sites. | # Ensure that there is L3 connectivity between all the sites. | ||
| - | # Ensure that provider / WAN network supports IP Multicast | + | # Ensure that provider / WAN network supports IP Multicast. For IP Multicast, ASM / Bidir need to be supported for control plane communication and SSM for any data multicast traffic between sites. |
# Identify number and placement of OTV edge devices in each site. Remember that at FCS, L3 gateway and OTV edge for a VLAN can not reside on the same logical device. This may require provisioning a VDC off aggregation layer N7Ks to serve as OTV edge device. | # Identify number and placement of OTV edge devices in each site. Remember that at FCS, L3 gateway and OTV edge for a VLAN can not reside on the same logical device. This may require provisioning a VDC off aggregation layer N7Ks to serve as OTV edge device. | ||
# Identify '''internal interface''' L2 connectivity of each OTV edge device to aggregation layer. Will L2 connection be single-homed to one aggregation switch, multi-homed using STP or multi-homed with VPC? | # Identify '''internal interface''' L2 connectivity of each OTV edge device to aggregation layer. Will L2 connection be single-homed to one aggregation switch, multi-homed using STP or multi-homed with VPC? | ||
| - | # Identify L3 connectivity of each OTV edge device to the provider network / WAN. Will L3 connection be single-homed or multi-homed connecting to 2 or more WAN endpoints? In multi-homed case, consider that there can only be 1 '''join-interface''' | + | # Identify L3 connectivity of each OTV edge device to the provider network / WAN. Will L3 connection be single-homed or multi-homed connecting to 2 or more WAN endpoints? In multi-homed case, consider that there can only be 1 '''join-interface'''. |
# Decide which VLANs will be extended and how many Overlays will be used for that. Most simple design can just use 1 Overlay, however a more complex design can be used with VLANs split between Overlays for loadbalancing. | # Decide which VLANs will be extended and how many Overlays will be used for that. Most simple design can just use 1 Overlay, however a more complex design can be used with VLANs split between Overlays for loadbalancing. | ||
| - | # Note if any of VLANs being extended are running FHRPs on L3 gateways. If multiple sites have FHRP running for VLAN being extended, '''HSRP localization''' feature | + | # Note if any of the VLANs being extended are running FHRPs on L3 gateways. If multiple sites have FHRP running for VLAN being extended, '''HSRP localization''' feature should be used. |
| - | # Designate a '''site VLAN''' to be used for communication between 2 OTV edge devices in | + | # Designate a '''site VLAN''' to be used for communication between 2 OTV edge devices in the same site. In case sites ever get merged, it is recommended to use the same VLAN to provision a '''site VLAN''' in each site, regardless whether it currently has 1 or 2 '''edge devices'''. |
===Procedures=== | ===Procedures=== | ||
| - | These procedures | + | These procedures outline the configuration necessary for OTV to be enabled in each site. In this example, it is assumed that certain prerequisites (such as L3 connectivity and Multicast) are already configured. |
# Enable OTV feature. | # Enable OTV feature. | ||
| Line 394: | Line 399: | ||
</div></div> | </div></div> | ||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
===Additional Resources=== | ===Additional Resources=== | ||
For more detailed information on OTV, please see the following link:<br />[http://tools.ietf.org/html/draft-hasmit-otv-00 OTV IETF draft] | For more detailed information on OTV, please see the following link:<br />[http://tools.ietf.org/html/draft-hasmit-otv-00 OTV IETF draft] | ||
| + | |||
| + | [http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/otv/configuration/guide/b_Cisco_Nexus_7000_Series_NX-OS_OTV_Configuration_Guide__Release_5.x.html ''Cisco Nexus 7000 Series NX-OS OTV Configuration Guide, Release 5.x''] | ||
===Acronyms=== | ===Acronyms=== | ||
OTV - Overlay Transport Virualization<br /> ASM - Any Source Multicast<br /> DCI - Data Center Interconnect<br /> SSM - Source Specific Multicast<br /> WAN - Wide Area Network<br /> FCS - First Customer Ship<br /> SP - Service Provider | OTV - Overlay Transport Virualization<br /> ASM - Any Source Multicast<br /> DCI - Data Center Interconnect<br /> SSM - Source Specific Multicast<br /> WAN - Wide Area Network<br /> FCS - First Customer Ship<br /> SP - Service Provider | ||
| + | [[Category: Nexus 7000 Series Switches]] | ||
[[Category: Configuration Examples]] | [[Category: Configuration Examples]] | ||
Latest revision as of 19:12, 12 August 2010
| Test Details Test Details | |
| Goal of Test |
Outline OTV design and configuration steps in a methodical "cook-book" manner. |
| Data to Record | show otv show otv adjacency show otv route show otv vlan show mac address-table |
| Estimated Time Needed | 30 min (assuming all pre-requisites are met) |
Contents |
Overview
For additional information regarding Overlay Transport Virtualization (OTV) on Cisco NX-OS devices, refer to the overview chapter available in the Cisco Nexus 7000 Series NX-OS OTV Configuration Guide, Release 5.x available at:
Diagram
Click on thumbnail to enlarge:
Design & Prerequisites
With OTV, it is extremely important to ensure that you have a good network design and that your network meets all the pre-requisites to support OTV deployment:
- Network design - have clear understanding of boundaries between:
- L2 and L3 in each site
- Functional data center layers (access, aggregation, core) in each site
- Each site and provider / WAN network.
- Ensure that there is L3 connectivity between all the sites.
- Ensure that provider / WAN network supports IP Multicast. For IP Multicast, ASM / Bidir need to be supported for control plane communication and SSM for any data multicast traffic between sites.
- Identify number and placement of OTV edge devices in each site. Remember that at FCS, L3 gateway and OTV edge for a VLAN can not reside on the same logical device. This may require provisioning a VDC off aggregation layer N7Ks to serve as OTV edge device.
- Identify internal interface L2 connectivity of each OTV edge device to aggregation layer. Will L2 connection be single-homed to one aggregation switch, multi-homed using STP or multi-homed with VPC?
- Identify L3 connectivity of each OTV edge device to the provider network / WAN. Will L3 connection be single-homed or multi-homed connecting to 2 or more WAN endpoints? In multi-homed case, consider that there can only be 1 join-interface.
- Decide which VLANs will be extended and how many Overlays will be used for that. Most simple design can just use 1 Overlay, however a more complex design can be used with VLANs split between Overlays for loadbalancing.
- Note if any of the VLANs being extended are running FHRPs on L3 gateways. If multiple sites have FHRP running for VLAN being extended, HSRP localization feature should be used.
- Designate a site VLAN to be used for communication between 2 OTV edge devices in the same site. In case sites ever get merged, it is recommended to use the same VLAN to provision a site VLAN in each site, regardless whether it currently has 1 or 2 edge devices.
Procedures
These procedures outline the configuration necessary for OTV to be enabled in each site. In this example, it is assumed that certain prerequisites (such as L3 connectivity and Multicast) are already configured.
- Enable OTV feature.
- Create a logical Overlay interface.
- Configure IGMP v3 on join-interface.
- Configure join-interface (you will get a warning about IGMP v3 regardless of completion of STEP 3).
- Configure multicast groups for control traffic between OTV sites and for any multicast data traffic between OTV sites.
- Configure VLANs to be extended over this Overlay.
- Repeat steps 2-6 for any additional Overlays that need to be configured.
- Configure site VLAN and make sure that between 2 OTV edge devices in a single site, a L2 connectivity exists through that VLAN.
- Verify proper OTV operation.
Examples
Following contains configuration example from one of OTV edge devices:
!STEP 1: feature otv !STEP 2: interface Overlay1 !STEP 3: interface Ethernet1/9 ip igmp version 3 !STEP 4: otv join-interface Ethernet1/9 !STEP 5: otv control-group 239.1.1.1 otv data-group 232.1.1.0/24 !STEP 6: otv extend-vlan 110 !STEP 8: otv site-vlan 200
Verification of OTV operation and connectivity between multiple sites:
SITE 1
!STEP 9:
! From SITE 1, OTV EDGE 1:
show otv
OTV Overlay Information
Overlay Interface Overlay1
VPN Name : Overlay1
VPN ID : 230
State : UP
IPv4 multicast group : Overlay1-239.1.1.1
IPv6 multicast group : [None]
Mcast data group range(s): 232.1.1.0/24
External interface(s) : Ethernet1/9
External IPv4 address : 10.4.9.2
External IPv6 address : 0::
Encapsulation format : GRE/IPv4
Site-vlan : 200
Capability : Multicast-Reachable
Is Adjacency Server : NO
Adj Server Configured : NO
Prim/Sec Adj Svr(s) : [None] / [None]
OTV_EDGE1_SITE1# show otv adjacency
Overlay Adjacency database
Overlay-Interface Overlay1 :
System-ID Dest Addr Adj-State TM_State Up Time Adj-State
001b.54c2.43c1 10.3.8.2 default default 1w3d UP
001b.54c2.43c3 10.5.10.2 default default 1w3d UP
001b.54c2.43c4 10.7.11.2 default default 2d17h UP
OTV_EDGE1_SITE1# show otv isis adjacency
OTV-IS-IS process: default VPN: Overlay1
OTV-IS-IS adjacency database:
System ID SNPA Level State Hold Time Interface
N7010-I4-OTV_E 001b.54c2.43c1 1 UP 00:00:25 Overlay1
OTV_EDGE2_SITE 001b.54c2.43c3 1 UP 00:00:27 Overlay1
OTV_EDGE_SITE3 001b.54c2.43c4 1 UP 00:00:07 Overlay1
OTV_EDGE1_SITE1# show otv route
OTV Unicast MAC Routing Table For Overlay1
VLAN MAC-Address Metric Uptime LastUpdt Owner Next-hop(s)
!100 MACs from SITE 1 - local
110 0000.6e01.010a 1 2d16h 2d16h lmac port-channel1
110 0000.6e01.010b 1 2d16h 2d16h lmac port-channel1
...
110 0000.6e01.016c 1 2d16h 2d16h lmac port-channel1
110 0000.6e01.016d 1 2d16h 2d16h lmac port-channel1
!100 MACs from SITE 2
110 0000.6e02.020a 42 2d16h 2d16h isis_otv-default Overlay1-10.3.8.2
110 0000.6e02.020b 42 2d16h 2d16h isis_otv-default Overlay1-10.3.8.2
...
110 0000.6e02.026c 42 2d16h 2d16h isis_otv-default Overlay1-10.3.8.2
110 0000.6e02.026d 42 2d16h 2d16h isis_otv-default Overlay1-10.3.8.2
!100 MACs from SITE 3
110 0000.6e03.030a 42 2d16h 2d16h isis_otv-default Overlay1-10.7.11.2
110 0000.6e03.030b 42 2d16h 2d16h isis_otv-default Overlay1-10.7.11.2
...
110 0000.6e03.036c 42 2d16h 2d16h isis_otv-default Overlay1-10.7.11.2
110 0000.6e03.036d 42 2d16h 2d16h isis_otv-default Overlay1-10.7.11.2
OTV_EDGE1_SITE1# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
G - 001b.54c2.43c2 static - F F sup-eth1(R)
!100 MACs from SITE 1 - local
* 110 0000.0c07.ac6e dynamic 0 F F Po1
* 110 0000.6e01.010a dynamic 0 F F Po1
...
* 110 0000.6e01.016c dynamic 0 F F Po1
* 110 0000.6e01.016d dynamic 0 F F Po1
!100 MACs from SITE 2 learned via Overlay
O 110 0000.6e02.020a dynamic 0 F F Overlay1
O 110 0000.6e02.020b dynamic 0 F F Overlay1
...
O 110 0000.6e02.026c dynamic 0 F F Overlay1
O 110 0000.6e02.026d dynamic 0 F F Overlay1
!100 MACs from SITE 3 learned via Overlay
O 110 0000.6e03.030a dynamic 0 F F Overlay1
O 110 0000.6e03.030b dynamic 0 F F Overlay1
...
O 110 0000.6e03.036c dynamic 0 F F Overlay1
O 110 0000.6e03.036d dynamic 0 F F Overlay1
OTV_EDGE1_SITE1# show otv site
OTV Overlay Information
Site-VLAN : 200
Site Adjacency database
Overlay: Overlay1-239.1.1.1, Adjacencies: 2
System-ID Priority Ordinal
* 001b.54c2.43c2 0 0
001b.54c2.43c3 0 1
OTV_EDGE1_SITE1# show otv vlan
OTV VLAN Configuration Information
VLAN-ID VlanState Switchport/ External Overlay
Forward Count Interface Group
110 UP 1/1 Ethernet1/9 Overlay1-239.1.1.1
!Will only show up at AED - so we know EDGE 1 is AED
OTV_EDGE1_SITE1# show otv vlan auth
OTV VLAN Configuration Information
VLAN-ID VlanState Switchport/ External Overlay
Forward Count Interface Group
110 UP 1/1 Ethernet1/9 Overlay1-239.1.1.1
OTV_EDGE1_SITE1# show otv arp
OTV ARP/ND L3->L2 Address Mapping Cache
Overlay Interface Overlay1
VLAN/MAC Address Uptime Layer-3 Address Exp Time Left
0110-001b.54c2.4c41 2w4d 110.2.2.1 00:19:55
0110-0000.6e02.020a 00:06:05 110.2.2.10 00:13:54
0110-0000.6e02.020b 00:06:05 110.2.2.11 00:13:54
0110-0000.6e02.020c 00:06:05 110.2.2.12 00:13:54
...
0110-0000.6e02.026c 00:06:05 110.2.2.108 00:13:54
0110-0000.6e02.026d 00:06:05 110.2.2.109 00:13:54
0110-0000.6e03.030a 00:06:06 110.3.3.10 00:13:53
0110-0000.6e03.030b 00:06:06 110.3.3.11 00:13:53
0110-0000.6e03.030c 00:06:05 110.3.3.12 00:13:54
...
0110-0000.6e03.0348 00:06:05 110.3.3.72 00:13:54
0110-0000.6e03.0349 00:06:05 110.3.3.73 00:13:54
0110-0000.6e03.034a 00:06:05 110.3.3.74 00:13:54
0110-0000.6e03.034b 00:06:05 110.3.3.75 00:13:54
SITE 3
!On SITE 3
OTV_EDGE_SITE3#
show otv
OTV Overlay Information
Overlay Interface Overlay1
VPN Name : Overlay1
VPN ID : 245
State : UP
IPv4 multicast group : Overlay1-239.1.1.1
IPv6 multicast group : [None]
Mcast data group range(s): 232.1.1.0/24
External interface(s) : Ethernet1/17
External IPv4 address : 10.7.11.2
External IPv6 address : 0::
Encapsulation format : GRE/IPv4
Site-vlan : 1
Capability : Multicast-Reachable
Is Adjacency Server : NO
Adj Server Configured : NO
Prim/Sec Adj Svr(s) : [None] / [None]
OTV_EDGE_SITE3#
show otv adjacency
Overlay Adjacency database
Overlay-Interface Overlay1 :
System-ID Dest Addr Adj-State TM_State Up Time Adj-State
001b.54c2.43c1 10.3.8.2 default default 2d17h UP
001b.54c2.43c2 10.4.9.2 default default 2d17h UP
001b.54c2.43c3 10.5.10.2 default default 2d17h UP
OTV_EDGE_SITE3#
show otv arp
OTV ARP/ND L3->L2 Address Mapping Cache
Overlay Interface Overlay1
VLAN/MAC Address Uptime Layer-3 Address Exp Time Left
0110-001b.54c2.39c1 3d05h 110.1.1.2 00:19:55
0110-001b.54c2.8541 2w3d 110.1.1.3 00:19:55
0110-0000.6e01.010a 00:06:06 110.1.1.10 00:13:53
0110-0000.6e01.010b 00:06:06 110.1.1.11 00:13:53
0110-0000.6e01.010c 00:06:06 110.1.1.12 00:13:53
show otv isis adjacency
OTV-IS-IS process: default VPN: Overlay1
OTV-IS-IS adjacency database:
System ID SNPA Level State Hold Time Interface
N7010-I4-OTV_E 001b.54c2.43c1 1 UP 00:00:29 Overlay1
OTV_EDGE1_SITE 001b.54c2.43c2 1 UP 00:00:31 Overlay1
OTV_EDGE2_SITE 001b.54c2.43c3 1 UP 00:00:29 Overlay1
OTV_EDGE_SITE3# show otv route
OTV Unicast MAC Routing Table For Overlay1
VLAN MAC-Address Metric Uptime LastUpdt Owner Next-hop(s)
!100 MACs from SITE 1
110 0000.6e01.010a 42 2d16h 2d16h isis_otv-default Overlay1-10.4.9.2
110 0000.6e01.010b 42 2d16h 2d16h isis_otv-default Overlay1-10.4.9.2
...
110 0000.6e01.016c 42 2d16h 2d16h isis_otv-default Overlay1-10.4.9.2
110 0000.6e01.016d 42 2d16h 2d16h isis_otv-default Overlay1-10.4.9.2
!100 MACs from SITE 2
110 0000.6e02.020a 42 2d16h 2d16h isis_otv-default Overlay1-10.3.8.2
110 0000.6e02.020b 42 2d16h 2d16h isis_otv-default Overlay1-10.3.8.2
...
110 0000.6e02.026c 42 2d16h 2d16h isis_otv-default Overlay1-10.3.8.2
110 0000.6e02.026d 42 2d16h 2d16h isis_otv-default Overlay1-10.3.8.2
!100 MACs from SITE 3 - local
110 0000.6e03.030a 1 2d16h 2d16h lmac Ethernet1/19
110 0000.6e03.030b 1 2d16h 2d16h lmac Ethernet1/19
...
110 0000.6e03.036c 1 2d16h 2d16h lmac Ethernet1/19
110 0000.6e03.036d 1 2d16h 2d16h lmac Ethernet1/19
OTV_EDGE_SITE3#
show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
!100 MACs from SITE 1
O 110 0000.6e01.010a dynamic 0 F F Overlay1
O 110 0000.6e01.010b dynamic 0 F F Overlay1
...
O 110 0000.6e01.016c dynamic 0 F F Overlay1
O 110 0000.6e01.016d dynamic 0 F F Overlay1
!100 MACs from SITE 2
O 110 0000.6e02.020a dynamic 0 F F Overlay1
O 110 0000.6e02.020b dynamic 0 F F Overlay1
...
O 110 0000.6e02.026c dynamic 0 F F Overlay1
O 110 0000.6e02.026d dynamic 0 F F Overlay1
!100 MACs from SITE 3 - local
* 110 0000.6e03.030a dynamic 0 F F Eth1/19
* 110 0000.6e03.030b dynamic 0 F F Eth1/19
...
* 110 0000.6e03.036c dynamic 0 F F Eth1/19
* 110 0000.6e03.036d dynamic 0 F F Eth1/19
OTV_EDGE_SITE3# show otv site
OTV Overlay Information
Site-VLAN : 1
Site Adjacency database
Overlay: Overlay1-239.1.1.1, Adjacencies: 1
System-ID Priority Ordinal
* 001b.54c2.43c4 0 0
OTV_EDGE_SITE3#
show otv vlan
OTV VLAN Configuration Information
VLAN-ID VlanState Switchport/ External Overlay
Forward Count Interface Group
110 UP 1/1 Ethernet1/17 Overlay1-239.1.1.1
OTV_EDGE_SITE3#
show otv vlan auth
OTV VLAN Configuration Information
VLAN-ID VlanState Switchport/ External Overlay
Forward Count Interface Group
110 UP 1/1 Ethernet1/17 Overlay1-239.1.1.1
Additional Resources
For more detailed information on OTV, please see the following link:
OTV IETF draft
Cisco Nexus 7000 Series NX-OS OTV Configuration Guide, Release 5.x
Acronyms
OTV - Overlay Transport Virualization
ASM - Any Source Multicast
DCI - Data Center Interconnect
SSM - Source Specific Multicast
WAN - Wide Area Network
FCS - First Customer Ship
SP - Service Provider