Openstack with LISP-enabled OpenVSwitch
From DocWiki
m (→Installing the LISP-enabled OVS bridge) |
m (→Installing the LISP-enabled OVS bridge) |
||
Line 171: | Line 171: | ||
which should yield an output similar to the following: | which should yield an output similar to the following: | ||
<pre> | <pre> | ||
- | 1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server | + | 1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server |
1409 ? 00:00:00 ovs-vswitchd | 1409 ? 00:00:00 ovs-vswitchd | ||
1410 ? 00:00:00 ovs-vswitchd | 1410 ? 00:00:00 ovs-vswitchd |
Revision as of 11:29, 2 December 2012
This wiki describes all the steps necessary to setup a testbed of Openstack Folsom nodes (All-in-one node and Compute nodes), which rely on a LISP-enabled OpenVSwitch (OVS) bridge.
cisco ("public") Network ----+----------------------+----- | | +--------+--------+ +-------+--------+ | All-In-One Node | | Compute Node | | | | | | | | | | Nova | | Nova -agent | | Keystone | | Nova-compute | | Glance | | Nova-Network | | Nova-Network | +----------------+ | Cinder | | Horizon | +-----------------+
Contents |
Testbed Setup
The Openstack nodes are considered hereafter to be Ubuntu 12.04.1 LTS (Precise Pangolin) machines (running the 3.2.0-29 kernel version). The following dependencies need to be met.
sudo apt-get update sudo apt-get upgrade (NOT dist-upgrade) sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \ clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd sudo apt-get install linux-headers-3.2.0-29-generic # For KVM sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
In case you wish to also use Wireshark (with LISP dissector included):
$ sudo apt-get build-dep wireshark $ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev $ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2 $ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp $ cd /tmp/wireshark-1.8.3/ $ ./autogen.sh $ ./configure --enable-setcap-install $ make $ sudo make install $ sudo ldconfig $ wireshark &
Network-wise:
Public + Management Network --+--------------------------+----- | | | | 10.10.10.10 10.10.11.10 | | +------|-------------+ +--------|-------+ | (eth0) | | (eth0) | | | | | | | | | | | | | | [ br-lisp ] | | [br-lisp] | | | | | | | | | (vnet0) (vnet1) | | (vnet0) | | | | | | | | | VM VM | | VM | | 192.168. 192.168. | | 192.168. | | 127.2 127.4 | | 127.3 | +--------------------+ +----------------+ All-In-One Node Compute Node [] OVS-LISP bridge () Network interface
Installing the LISP-enabled OVS bridge
On both machines do the following. First download the corresponding code:
cd /home/user/ git clone https://
First install lispmob:
cd /home/user/lispmob-ovs/ make sudo make install sudo depmod -a
Afterward install the modified LISP-enabled OpenVswitch:
cd /home/user/ovs-lisp/ ./boot.sh ./configure --with-linux=/lib/modules/`uname -r`/build make
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/ sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
...and remove 'Module.symvers' from the 'clean' target (on line 33)
sudo reboot
Then, you will need to REMAKE and REINSTALL lisp
cd /home/lakafosi/lispmob-ovs make sudo make install sudo depmod -a cd /home/lakafosi/ovs-lisp/ make sudo make install sudo depmod -a sudo dmesg -c
In order to bring up the OVS bridge:
sudo /home/user/ovs-scripts/ovs-start
which includes:
#!/bin/bash BUILD_DIR=/home/lakafosi/ovs-lisp OVS_PATH=/usr/local rmmod -s openvswitch modprobe lisp insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko mkdir -p ${OVS_PATH}/etc/openvswitch rm ${OVS_PATH}/etc/openvswitch/conf.db ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema ${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ --remote=db:Open_vSwitch,manager_options \ --private-key=db:SSL,private_key \ --certificate=db:SSL,certificate \ --bootstrap-ca-cert=db:SSL,ca_cert \ --pidfile --detach ${OVS_PATH}/bin/ovs-vsctl --no-wait init ${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
You can verify everything is setup correctly by issuing:
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
which should yield an output similar to the following:
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server 1409 ? 00:00:00 ovs-vswitchd 1410 ? 00:00:00 ovs-vswitchd lisp 41286 1 openvswitch
Openstack
All-In-One Openstack Node
#!/bin/bash echo "=== lakafosi: Stopping firewall and allowing everyone... ===" iptables -F iptables -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X iptables -P INPUT ACCEPT iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT echo "=== lakafosi: Fixing the bridges... ===" VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}') #Alternatively: #VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i) echo "MAC address of guest VM: $VM_mac" brctl delif br100 vnet0 brctl delif br100 eth0 #ifconfig br100 0.0.0.0 #ifconfig br100 0.0.0.0 #ifconfig br100 192.168.127.1 netmask 255.255.255.0 ifconfig br100 down brctl delbr br100 ovs-vsctl -- --if-exists del-br br-lisp ovs-vsctl add-br br-lisp ovs-vsctl add-port br-lisp vnet0 ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.32.164.175 #AllInOne #ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.32.164.133 #Compute #ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.32.164.132 ovs-vsctl add-port br-lisp eth0 ovs-ofctl del-flows br-lisp #ovs-ofctl add-flows br-lisp /home/stack/Downloads/lisp_flow_table-AllInOne ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2 ovs-ofctl add-flow br-lisp priority=0,action=NORMAL ifconfig vnet0 up ifconfig eth0 0.0.0.0 #AllInOne ifconfig br-lisp 10.32.164.132 netmask 255.255.255.192 #Compute #ifconfig br-lisp 10.32.164.133 netmask 255.255.255.192 route add default gw 10.32.164.129 killall farpd farpd -i br-lisp 192.168.127.5 echo "=== lakafosi: Bridge Status: ===" brctl show ovs-vsctl show ovs-appctl fdb/show br-lisp ovs-dpctl show ovs-ofctl dump-flows br-lisp echo "=== lakafosi: Go and START lisp...! ===" #echo "lakafosi: Starting lisp..." #lispd -f /etc/lispd.conf & echo "=== lakafosi: lisp status: ===" ps -ea | grep lisp; lsmod | grep lisp