Nexus 7000 - OTV - Design and Configuration Example

From DocWiki

(Difference between revisions)
Jump to: navigation, search
(Additional Resources)
(Procedures)
(3 intermediate revisions not shown)
Line 12: Line 12:
| 30 min (assuming all pre-requisites are met)
| 30 min (assuming all pre-requisites are met)
|}
|}
 +
===Overview===
 +
For additional information regarding Overlay Transport Virtualization (OTV) on Cisco NX-OS devices, refer to the overview chapter available in the ''Cisco Nexus 7000 Series NX-OS OTV Configuration Guide, Release 5.x'' available at:
 +
 +
http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/otv/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_OTV_Configuration_Guide__Release_5.x_chapter1.html
 +
===Diagram===
===Diagram===
Line 36: Line 41:
## Each site and provider / WAN network.
## Each site and provider / WAN network.
# Ensure that there is L3 connectivity between all the sites.
# Ensure that there is L3 connectivity between all the sites.
-
# Ensure that provider / WAN network supports IP Multicast or decide which OTV edge device will act as adjacency server (check with DCBU on feature availability post-FCS of OTV). For IP Multicast, ASM / Bidir need to be supported for control plane communication and SSM for any data multicast traffic between sites.
+
# Ensure that provider / WAN network supports IP Multicast. For IP Multicast, ASM / Bidir need to be supported for control plane communication and SSM for any data multicast traffic between sites.
# Identify number and placement of OTV edge devices in each site. Remember that at FCS, L3 gateway and OTV edge for a VLAN can not reside on the same logical device. This may require provisioning a VDC off aggregation layer N7Ks to serve as OTV edge device.
# Identify number and placement of OTV edge devices in each site. Remember that at FCS, L3 gateway and OTV edge for a VLAN can not reside on the same logical device. This may require provisioning a VDC off aggregation layer N7Ks to serve as OTV edge device.
# Identify '''internal interface''' L2 connectivity of each OTV edge device to aggregation layer. Will L2 connection be single-homed to one aggregation switch, multi-homed using STP or multi-homed with VPC?
# Identify '''internal interface''' L2 connectivity of each OTV edge device to aggregation layer. Will L2 connection be single-homed to one aggregation switch, multi-homed using STP or multi-homed with VPC?
-
# Identify L3 connectivity of each OTV edge device to the provider network / WAN. Will L3 connection be single-homed or multi-homed connecting to 2 or more WAN endpoints? In multi-homed case, consider that there can only be 1 '''join-interface''' and based on that , decide whether join interface will be a MCEC or one of the uplinks. If only one of uplinks is used as join-interface, other one can be used for unicast traffic loadbalancing between sites.
+
# Identify L3 connectivity of each OTV edge device to the provider network / WAN. Will L3 connection be single-homed or multi-homed connecting to 2 or more WAN endpoints? In multi-homed case, consider that there can only be 1 '''join-interface'''.  
# Decide which VLANs will be extended and how many Overlays will be used for that. Most simple design can just use 1 Overlay, however a more complex design can be used with VLANs split between Overlays for loadbalancing.
# Decide which VLANs will be extended and how many Overlays will be used for that. Most simple design can just use 1 Overlay, however a more complex design can be used with VLANs split between Overlays for loadbalancing.
-
# Note if any of VLANs being extended are running FHRPs on L3 gateways. If multiple sites have FHRP running for VLAN being extended, '''HSRP localization''' feature (post-FCS) or other workarounds should be used!
+
# Note if any of the VLANs being extended are running FHRPs on L3 gateways. If multiple sites have FHRP running for VLAN being extended, '''HSRP localization''' feature should be used.
-
# Designate a '''site VLAN''' to be used for communication between 2 OTV edge devices in a single site. In case sites ever get merged, it is recommended to use same VLAN ID to provision a '''site VLAN''' in each site, regardless whether it currently has 1 or 2 '''edge devices'''.
+
# Designate a '''site VLAN''' to be used for communication between 2 OTV edge devices in the same site. In case sites ever get merged, it is recommended to use the same VLAN to provision a '''site VLAN''' in each site, regardless whether it currently has 1 or 2 '''edge devices'''.
===Procedures===
===Procedures===
-
These procedures will outline the configuration necessary for OTV to be enabled in each site. It is assumed that pre-requisites such as L3 connectivity and Multicast have already been configured and best practices for those have been used.
+
These procedures outline the configuration necessary for OTV to be enabled in each site. In this example, it is assumed that certain prerequisites (such as L3 connectivity and Multicast) are already configured.
# Enable OTV feature.
# Enable OTV feature.

Revision as of 19:12, 12 August 2010

Test Details Test Details
Goal of Test

Outline OTV design and configuration steps in a methodical "cook-book" manner.

Data to Record show otv
show otv adjacency
show otv route
show otv vlan
show mac address-table
Estimated Time Needed 30 min (assuming all pre-requisites are met)

Contents

Overview

For additional information regarding Overlay Transport Virtualization (OTV) on Cisco NX-OS devices, refer to the overview chapter available in the Cisco Nexus 7000 Series NX-OS OTV Configuration Guide, Release 5.x available at:

http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/otv/configuration/guide/Cisco_Nexus_7000_Series_NX-OS_OTV_Configuration_Guide__Release_5.x_chapter1.html

Diagram

Click on thumbnail to enlarge:

OTV+diagram.jpg











Design & Prerequisites

With OTV, it is extremely important to ensure that you have a good network design and that your network meets all the pre-requisites to support OTV deployment:

  1. Network design - have clear understanding of boundaries between:
    1. L2 and L3 in each site
    2. Functional data center layers (access, aggregation, core) in each site
    3. Each site and provider / WAN network.
  2. Ensure that there is L3 connectivity between all the sites.
  3. Ensure that provider / WAN network supports IP Multicast. For IP Multicast, ASM / Bidir need to be supported for control plane communication and SSM for any data multicast traffic between sites.
  4. Identify number and placement of OTV edge devices in each site. Remember that at FCS, L3 gateway and OTV edge for a VLAN can not reside on the same logical device. This may require provisioning a VDC off aggregation layer N7Ks to serve as OTV edge device.
  5. Identify internal interface L2 connectivity of each OTV edge device to aggregation layer. Will L2 connection be single-homed to one aggregation switch, multi-homed using STP or multi-homed with VPC?
  6. Identify L3 connectivity of each OTV edge device to the provider network / WAN. Will L3 connection be single-homed or multi-homed connecting to 2 or more WAN endpoints? In multi-homed case, consider that there can only be 1 join-interface.
  7. Decide which VLANs will be extended and how many Overlays will be used for that. Most simple design can just use 1 Overlay, however a more complex design can be used with VLANs split between Overlays for loadbalancing.
  8. Note if any of the VLANs being extended are running FHRPs on L3 gateways. If multiple sites have FHRP running for VLAN being extended, HSRP localization feature should be used.
  9. Designate a site VLAN to be used for communication between 2 OTV edge devices in the same site. In case sites ever get merged, it is recommended to use the same VLAN to provision a site VLAN in each site, regardless whether it currently has 1 or 2 edge devices.

Procedures

These procedures outline the configuration necessary for OTV to be enabled in each site. In this example, it is assumed that certain prerequisites (such as L3 connectivity and Multicast) are already configured.

  1. Enable OTV feature.
  2. Create a logical Overlay interface.
  3. Configure IGMP v3 on join-interface.
  4. Configure join-interface (you will get a warning about IGMP v3 regardless of completion of STEP 3).
  5. Configure multicast groups for control traffic between OTV sites and for any multicast data traffic between OTV sites.
  6. Configure VLANs to be extended over this Overlay.
  7. Repeat steps 2-6 for any additional Overlays that need to be configured.
  8. Configure site VLAN and make sure that between 2 OTV edge devices in a single site, a L2 connectivity exists through that VLAN.
  9. Verify proper OTV operation.

Examples

Following contains configuration example from one of OTV edge devices:

!STEP 1:
 
 feature otv
 
 !STEP 2:
 interface Overlay1
 
 !STEP 3:
 interface Ethernet1/9
   ip igmp version 3
  
 !STEP 4:
  otv join-interface Ethernet1/9
 
 !STEP 5:
   otv control-group 239.1.1.1
   otv data-group 232.1.1.0/24
 
 !STEP 6:
   otv extend-vlan 110
 
 !STEP 8:
 otv site-vlan 200
 

Verification of OTV operation and connectivity between multiple sites:

SITE 1

!STEP 9:
 ! From SITE 1, OTV EDGE 1:
 show otv
 
 OTV Overlay Information
 
 Overlay Interface Overlay1
  VPN Name                 : Overlay1
  VPN ID                   : 230
  State                    : UP
  IPv4 multicast group     : Overlay1-239.1.1.1
  IPv6 multicast group     : [None]
  Mcast data group range(s): 232.1.1.0/24
  External interface(s)    : Ethernet1/9
  External IPv4 address    : 10.4.9.2
  External IPv6 address    : 0::
  Encapsulation format     : GRE/IPv4
  Site-vlan                : 200
  Capability               : Multicast-Reachable
  Is Adjacency Server      : NO
  Adj Server Configured    : NO
  Prim/Sec Adj Svr(s)      : [None] / [None]
 
 OTV_EDGE1_SITE1# show otv adjacency
 
 Overlay Adjacency database
 Overlay-Interface Overlay1  :
 System-ID        Dest Addr        Adj-State TM_State  Up Time   Adj-State
 001b.54c2.43c1   10.3.8.2         default   default   1w3d      UP
 001b.54c2.43c3   10.5.10.2        default   default   1w3d      UP
 001b.54c2.43c4   10.7.11.2        default   default   2d17h     UP       

 OTV_EDGE1_SITE1# show otv isis adjacency
 
 OTV-IS-IS process: default VPN: Overlay1
 OTV-IS-IS adjacency database:
 System ID       SNPA            Level  State  Hold Time  Interface
 N7010-I4-OTV_E  001b.54c2.43c1  1      UP     00:00:25   Overlay1
 OTV_EDGE2_SITE  001b.54c2.43c3  1      UP     00:00:27   Overlay1
 OTV_EDGE_SITE3  001b.54c2.43c4  1      UP     00:00:07   Overlay1
 
 
 OTV_EDGE1_SITE1# show otv route
 OTV Unicast MAC Routing Table For Overlay1
 
 VLAN MAC-Address    Metric Uptime   LastUpdt Owner              Next-hop(s)
 !100 MACs from SITE 1 - local
 110  0000.6e01.010a 1      2d16h    2d16h    lmac               port-channel1
 110  0000.6e01.010b 1      2d16h    2d16h    lmac               port-channel1
 ...
 110  0000.6e01.016c 1      2d16h    2d16h    lmac               port-channel1
 110  0000.6e01.016d 1      2d16h    2d16h    lmac               port-channel1
 
 !100 MACs from SITE 2
 110  0000.6e02.020a 42     2d16h    2d16h    isis_otv-default   Overlay1-10.3.8.2
 110  0000.6e02.020b 42     2d16h    2d16h    isis_otv-default   Overlay1-10.3.8.2
 ...
 110  0000.6e02.026c 42     2d16h    2d16h    isis_otv-default   Overlay1-10.3.8.2
 110  0000.6e02.026d 42     2d16h    2d16h    isis_otv-default   Overlay1-10.3.8.2
 
 !100 MACs from SITE 3
 110  0000.6e03.030a 42     2d16h    2d16h    isis_otv-default   Overlay1-10.7.11.2
 110  0000.6e03.030b 42     2d16h    2d16h    isis_otv-default   Overlay1-10.7.11.2
 ...
 110  0000.6e03.036c 42     2d16h    2d16h    isis_otv-default   Overlay1-10.7.11.2
 110  0000.6e03.036d 42     2d16h    2d16h    isis_otv-default   Overlay1-10.7.11.2
 
 
 OTV_EDGE1_SITE1# show mac address-table
 
 Legend:
         * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
         age - seconds since last seen,+ - primary entry using vPC Peer-Link
    VLAN     MAC Address      Type      age     Secure NTFY    Ports
 ---------+-----------------+--------+---------+------+----+------------------
 G     -    001b.54c2.43c2    static       -       F    F  sup-eth1(R)
 !100 MACs from SITE 1 - local
 * 110      0000.0c07.ac6e    dynamic   0          F    F  Po1
 * 110      0000.6e01.010a    dynamic   0          F    F  Po1
 ...
 * 110      0000.6e01.016c    dynamic   0          F    F  Po1
 * 110      0000.6e01.016d    dynamic   0          F    F  Po1
 
 !100 MACs from SITE 2 learned via Overlay
 O 110      0000.6e02.020a    dynamic   0          F    F  Overlay1
 O 110      0000.6e02.020b    dynamic   0          F    F  Overlay1
 ...
 O 110      0000.6e02.026c    dynamic   0          F    F  Overlay1
 O 110      0000.6e02.026d    dynamic   0          F    F  Overlay1
 
 !100 MACs from SITE 3 learned via Overlay
 O 110      0000.6e03.030a    dynamic   0          F    F  Overlay1
 O 110      0000.6e03.030b    dynamic   0          F    F  Overlay1
 ...
 O 110      0000.6e03.036c    dynamic   0          F    F  Overlay1
 O 110      0000.6e03.036d    dynamic   0          F    F  Overlay1
 
 
 OTV_EDGE1_SITE1# show otv site
 OTV Overlay Information
 Site-VLAN                : 200
 Site Adjacency database
 Overlay: Overlay1-239.1.1.1, Adjacencies: 2
   System-ID                   Priority    Ordinal
 * 001b.54c2.43c2                     0          0
   001b.54c2.43c3                     0          1
 
 OTV_EDGE1_SITE1# show otv vlan
 OTV VLAN Configuration Information
 VLAN-ID  VlanState           Switchport/    External       Overlay
                              Forward Count  Interface      Group
 110      UP                  1/1            Ethernet1/9    Overlay1-239.1.1.1
 
  !Will only show up at AED - so we know EDGE 1 is AED
 OTV_EDGE1_SITE1# show otv vlan auth
 
 OTV VLAN Configuration Information
 VLAN-ID  VlanState           Switchport/    External       Overlay
                              Forward Count  Interface      Group
 110      UP                  1/1            Ethernet1/9    Overlay1-239.1.1.1
 
 OTV_EDGE1_SITE1# show otv arp
 
 OTV ARP/ND L3->L2 Address Mapping Cache
 Overlay Interface Overlay1
 VLAN/MAC Address      Uptime    Layer-3 Address     Exp Time Left
 0110-001b.54c2.4c41   2w4d      110.2.2.1           00:19:55
 0110-0000.6e02.020a   00:06:05  110.2.2.10          00:13:54
 0110-0000.6e02.020b   00:06:05  110.2.2.11          00:13:54
 0110-0000.6e02.020c   00:06:05  110.2.2.12          00:13:54
 ...
 0110-0000.6e02.026c   00:06:05  110.2.2.108         00:13:54
 0110-0000.6e02.026d   00:06:05  110.2.2.109         00:13:54
 0110-0000.6e03.030a   00:06:06  110.3.3.10          00:13:53
 0110-0000.6e03.030b   00:06:06  110.3.3.11          00:13:53
 0110-0000.6e03.030c   00:06:05  110.3.3.12          00:13:54
 ...
 0110-0000.6e03.0348   00:06:05  110.3.3.72          00:13:54
 0110-0000.6e03.0349   00:06:05  110.3.3.73          00:13:54
 0110-0000.6e03.034a   00:06:05  110.3.3.74          00:13:54
 0110-0000.6e03.034b   00:06:05  110.3.3.75          00:13:54
 
 

SITE 3

!On SITE 3
 
 OTV_EDGE_SITE3#
 show otv
 
 OTV Overlay Information
 
 Overlay Interface Overlay1
  VPN Name                 : Overlay1
  VPN ID                   : 245
  State                    : UP
  IPv4 multicast group     : Overlay1-239.1.1.1
  IPv6 multicast group     : [None]
  Mcast data group range(s): 232.1.1.0/24
  External interface(s)    : Ethernet1/17
  External IPv4 address    : 10.7.11.2
  External IPv6 address    : 0::
  Encapsulation format     : GRE/IPv4
  Site-vlan                : 1
  Capability               : Multicast-Reachable
  Is Adjacency Server      : NO
  Adj Server Configured    : NO
  Prim/Sec Adj Svr(s)      : [None] / [None]
 
 
 
 OTV_EDGE_SITE3#
 show otv adjacency
 
 Overlay Adjacency database
 
 Overlay-Interface Overlay1  :
 System-ID        Dest Addr        Adj-State TM_State  Up Time   Adj-State
 001b.54c2.43c1   10.3.8.2         default   default   2d17h     UP
 001b.54c2.43c2   10.4.9.2         default   default   2d17h     UP
 001b.54c2.43c3   10.5.10.2        default   default   2d17h     UP
 
 OTV_EDGE_SITE3#
 show otv arp
 
 OTV ARP/ND L3->L2 Address Mapping Cache
 Overlay Interface Overlay1
 VLAN/MAC Address      Uptime    Layer-3 Address     Exp Time Left
 0110-001b.54c2.39c1   3d05h     110.1.1.2           00:19:55
 0110-001b.54c2.8541   2w3d      110.1.1.3           00:19:55
 0110-0000.6e01.010a   00:06:06  110.1.1.10          00:13:53
 0110-0000.6e01.010b   00:06:06  110.1.1.11          00:13:53
 0110-0000.6e01.010c   00:06:06  110.1.1.12          00:13:53
 
 show otv isis adjacency
 
 OTV-IS-IS process: default VPN: Overlay1
 OTV-IS-IS adjacency database:
 System ID       SNPA            Level  State  Hold Time  Interface
 N7010-I4-OTV_E  001b.54c2.43c1  1      UP     00:00:29   Overlay1
 OTV_EDGE1_SITE  001b.54c2.43c2  1      UP     00:00:31   Overlay1
 OTV_EDGE2_SITE  001b.54c2.43c3  1      UP     00:00:29   Overlay1
 
 
 OTV_EDGE_SITE3# show otv route
 
 OTV Unicast MAC Routing Table For Overlay1
 
 VLAN MAC-Address    Metric Uptime   LastUpdt Owner              Next-hop(s)
 !100 MACs from SITE 1
 110  0000.6e01.010a 42     2d16h    2d16h    isis_otv-default   Overlay1-10.4.9.2
 110  0000.6e01.010b 42     2d16h    2d16h    isis_otv-default   Overlay1-10.4.9.2
 ...
 110  0000.6e01.016c 42     2d16h    2d16h    isis_otv-default   Overlay1-10.4.9.2
 110  0000.6e01.016d 42     2d16h    2d16h    isis_otv-default   Overlay1-10.4.9.2
 
 !100 MACs from SITE 2
 110  0000.6e02.020a 42     2d16h    2d16h    isis_otv-default   Overlay1-10.3.8.2
 110  0000.6e02.020b 42     2d16h    2d16h    isis_otv-default   Overlay1-10.3.8.2
 ...
 110  0000.6e02.026c 42     2d16h    2d16h    isis_otv-default   Overlay1-10.3.8.2
 110  0000.6e02.026d 42     2d16h    2d16h    isis_otv-default   Overlay1-10.3.8.2
 
 !100 MACs from SITE 3 - local
 110  0000.6e03.030a 1      2d16h    2d16h    lmac               Ethernet1/19
 110  0000.6e03.030b 1      2d16h    2d16h    lmac               Ethernet1/19
 ...
 110  0000.6e03.036c 1      2d16h    2d16h    lmac               Ethernet1/19
 110  0000.6e03.036d 1      2d16h    2d16h    lmac               Ethernet1/19
 
 
 OTV_EDGE_SITE3#
 show mac address-table
 
 Legend:
         * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
         age - seconds since last seen,+ - primary entry using vPC Peer-Link
    VLAN     MAC Address      Type      age     Secure NTFY    Ports
 ---------+-----------------+--------+---------+------+----+------------------
 !100 MACs from SITE 1
 O 110      0000.6e01.010a    dynamic   0          F    F  Overlay1
 O 110      0000.6e01.010b    dynamic   0          F    F  Overlay1
 ...
 O 110      0000.6e01.016c    dynamic   0          F    F  Overlay1
 O 110      0000.6e01.016d    dynamic   0          F    F  Overlay1
 
 !100 MACs from SITE 2
 O 110      0000.6e02.020a    dynamic   0          F    F  Overlay1
 O 110      0000.6e02.020b    dynamic   0          F    F  Overlay1
 ...
 O 110      0000.6e02.026c    dynamic   0          F    F  Overlay1
 O 110      0000.6e02.026d    dynamic   0          F    F  Overlay1
 
 !100 MACs from SITE 3 - local
 * 110      0000.6e03.030a    dynamic   0          F    F  Eth1/19
 * 110      0000.6e03.030b    dynamic   0          F    F  Eth1/19
 ...
 * 110      0000.6e03.036c    dynamic   0          F    F  Eth1/19
 * 110      0000.6e03.036d    dynamic   0          F    F  Eth1/19
 
 OTV_EDGE_SITE3# show otv site
 OTV Overlay Information
 
  Site-VLAN                : 1
 
 Site Adjacency database
 Overlay: Overlay1-239.1.1.1, Adjacencies: 1
   System-ID                   Priority    Ordinal
 * 001b.54c2.43c4                     0          0
 
 
 OTV_EDGE_SITE3#
 show otv vlan
 
 OTV VLAN Configuration Information
 VLAN-ID  VlanState           Switchport/    External       Overlay
                              Forward Count  Interface      Group
 
 110      UP                  1/1            Ethernet1/17   Overlay1-239.1.1.1
 
 
 OTV_EDGE_SITE3#
 show otv vlan auth
 
 OTV VLAN Configuration Information
 VLAN-ID  VlanState           Switchport/    External       Overlay
                              Forward Count  Interface      Group
 110      UP                  1/1            Ethernet1/17   Overlay1-239.1.1.1
 
 


Additional Resources

For more detailed information on OTV, please see the following link:
OTV IETF draft

Cisco Nexus 7000 Series NX-OS OTV Configuration Guide, Release 5.x

Acronyms

OTV - Overlay Transport Virualization
ASM - Any Source Multicast
DCI - Data Center Interconnect
SSM - Source Specific Multicast
WAN - Wide Area Network
FCS - First Customer Ship
SP - Service Provider

Rating: 4.5/5 (11 votes cast)

Personal tools