From DocWiki

Revision as of 11:29, 2 December 2012

This wiki describes all the steps necessary to setup a testbed of Openstack Folsom nodes (All-in-one node and Compute nodes), which rely on a LISP-enabled OpenVSwitch (OVS) bridge.

                   cisco ("public") Network
           ----+----------------------+-----
               |                      |
      +--------+--------+     +-------+--------+
      | All-In-One Node |     | Compute Node   |
      |                 |     |                |
      |                 |     |                |
      |  Nova           |     |  Nova -agent   |
      |  Keystone       |     |  Nova-compute  |
      |  Glance         |     |  Nova-Network  |
      |  Nova-Network   |     +----------------+
      |  Cinder         |
      |  Horizon        |
      +-----------------+

Testbed Setup

The Openstack nodes are considered hereafter to be Ubuntu 12.04.1 LTS (Precise Pangolin) machines (running the 3.2.0-29 kernel version). The following dependencies need to be met.

sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
 clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst

In case you wish to also use Wireshark (with LISP dissector included):

$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &

Network-wise:

         Public + Management Network
     --+--------------------------+----- 
       |                          |
       |                          |
  10.10.10.10                10.10.11.10
       |                          |
+------|-------------+   +--------|-------+
|    (eth0)          |   |      (eth0)    |
|      |             |   |        |       |
|      |             |   |        |       |
|  [ br-lisp ]       |   |     [br-lisp]  |
|    |       |       |   |        |       |
| (vnet0) (vnet1)    |   |     (vnet0)    |
|    |       |       |   |        |       |
|    VM      VM      |   |        VM      |
| 192.168. 192.168.  |   |     192.168.   |
|   127.2   127.4    |   |       127.3    |
+--------------------+   +----------------+
   All-In-One Node          Compute Node


[] OVS-LISP bridge
() Network interface

Installing the LISP-enabled OVS bridge

On both machines do the following. First download the corresponding code:

cd /home/user/
git clone https://

First install lispmob:

cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a

Afterward install the modified LISP-enabled OpenVswitch:

cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make

cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in

...and remove 'Module.symvers' from the 'clean' target (on line 33)

sudo reboot

Then, you will need to REMAKE and REINSTALL lisp

cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c

In order to bring up the OVS bridge:

sudo /home/user/ovs-scripts/ovs-start

which includes:

#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
                     --remote=db:Open_vSwitch,manager_options \
                     --private-key=db:SSL,private_key \
                     --certificate=db:SSL,certificate \
                     --bootstrap-ca-cert=db:SSL,ca_cert \
                     --pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs

You can verify everything is setup correctly by issuing:

ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs

which should yield an output similar to the following:

 1404 ?        00:00:00 ovs_workq 1406 ?        00:00:00 ovsdb-server
 1409 ?        00:00:00 ovs-vswitchd
 1410 ?        00:00:00 ovs-vswitchd
lisp                   41286  1 openvswitch

Openstack

All-In-One Openstack Node

#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.32.164.175
#AllInOne
#ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.32.164.133
#Compute
#ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.32.164.132
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
#ovs-ofctl add-flows br-lisp /home/stack/Downloads/lisp_flow_table-AllInOne
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.32.164.132 netmask 255.255.255.192
#Compute
#ifconfig br-lisp 10.32.164.133 netmask 255.255.255.192
route add default gw 10.32.164.129
killall farpd
farpd -i br-lisp 192.168.127.5
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp

Compute Openstack Node