Openstack with LISP-enabled OpenVSwitch
From DocWiki
(→All-In-One Openstack Node) |
(→Testbed Setup) |
||
Line 89: | Line 89: | ||
</pre> | </pre> | ||
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).'' | If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).'' | ||
+ | |||
+ | |||
+ | == Map Server (MS) == | ||
+ | |||
+ | Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments. | ||
+ | Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically. | ||
+ | |||
+ | <pre> | ||
+ | <?xml version="1.0" encoding="ISO-8859-1"?> | ||
+ | |||
+ | <db> | ||
+ | |||
+ | <eid prefix="0.0.0.0/3" act="2" ttl="5" /> | ||
+ | <eid prefix="128.0.0.0/8" act="2" ttl="5" /> | ||
+ | <eid prefix="171.0.0.0/8" act="2" ttl="5" /> | ||
+ | <eid prefix="224.0.0.0/4" act="2" ttl="5" /> | ||
+ | |||
+ | <eid prefix="192.168.127.3/32" ttl="1440" A="true"> | ||
+ | <rloc> | ||
+ | <address>10.10.10.10</address> | ||
+ | <priority>1</priority> | ||
+ | <weight>100</weight> | ||
+ | <m_priority>255</m_priority> | ||
+ | <m_weight>0</m_weight> | ||
+ | <reachable>true</reachable> | ||
+ | </rloc> | ||
+ | </eid> | ||
+ | |||
+ | <eid prefix="192.168.127.4/32" ttl="1440" A="true"> | ||
+ | <rloc> | ||
+ | <address>10.10.11.10</address> | ||
+ | <priority>1</priority> | ||
+ | <weight>100</weight> | ||
+ | <m_priority>255</m_priority> | ||
+ | <m_weight>0</m_weight> | ||
+ | <reachable>true</reachable> | ||
+ | </rloc> | ||
+ | </eid> | ||
+ | |||
+ | </db> | ||
+ | </pre> | ||
= Installing the LISP-enabled OVS bridge = | = Installing the LISP-enabled OVS bridge = |
Revision as of 11:57, 2 December 2012
This wiki describes all the steps necessary to setup a testbed of Openstack Folsom nodes (All-in-one node and Compute nodes), which rely on a LISP-enabled OpenVSwitch (OVS) bridge.
cisco ("public") Network ----+----------------------+----- | | +--------+--------+ +-------+--------+ | All-In-One Node | | Compute Node | | | | | | | | | | Nova | | Nova -agent | | Keystone | | Nova-compute | | Glance | | Nova-Network | | Nova-Network | +----------------+ | Cinder | | Horizon | +-----------------+
Contents |
Testbed Setup
Dependencies
The Openstack nodes are considered hereafter to be Ubuntu 12.04.1 LTS (Precise Pangolin) machines (running the 3.2.0-29 kernel version). The following dependencies need to be met.
sudo apt-get update sudo apt-get upgrade (NOT dist-upgrade) sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \ clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd sudo apt-get install linux-headers-3.2.0-29-generic # For KVM sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
In case you wish to also use Wireshark (with LISP dissector included):
$ sudo apt-get build-dep wireshark $ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev $ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2 $ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp $ cd /tmp/wireshark-1.8.3/ $ ./autogen.sh $ ./configure --enable-setcap-install $ make $ sudo make install $ sudo ldconfig $ wireshark &
Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
- 1.8GB for the All-In-One Node
- 1.2GB for the Compute Node
Network
Public + Management Network --+--------------------------+----- | | | | 10.10.10.10 10.10.11.10 | | +------|-------------+ +--------|-------+ | (eth0) | | (eth0) | | | | | | | | | | | | | | [ br-lisp ] | | [br-lisp] | | | | | | | | | (vnet0) (vnet1) | | (vnet0) | | | | | | | | | VM VM | | VM | | 192.168. 192.168. | | 192.168. | | 127.2 127.4 | | 127.3 | +--------------------+ +----------------+ All-In-One Node Compute Node [] OVS-LISP bridge () Network interface
If instead of physical machines you plan on setting this testbed up with VMware Fusion, make sure the network is configured as Bridged (Ethernet).
Map Server (MS)
Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments. Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
<?xml version="1.0" encoding="ISO-8859-1"?> <db> <eid prefix="0.0.0.0/3" act="2" ttl="5" /> <eid prefix="128.0.0.0/8" act="2" ttl="5" /> <eid prefix="171.0.0.0/8" act="2" ttl="5" /> <eid prefix="224.0.0.0/4" act="2" ttl="5" /> <eid prefix="192.168.127.3/32" ttl="1440" A="true"> <rloc> <address>10.10.10.10</address> <priority>1</priority> <weight>100</weight> <m_priority>255</m_priority> <m_weight>0</m_weight> <reachable>true</reachable> </rloc> </eid> <eid prefix="192.168.127.4/32" ttl="1440" A="true"> <rloc> <address>10.10.11.10</address> <priority>1</priority> <weight>100</weight> <m_priority>255</m_priority> <m_weight>0</m_weight> <reachable>true</reachable> </rloc> </eid> </db>
Installing the LISP-enabled OVS bridge
On both machines do the following. First download the corresponding code:
cd /home/user/ git clone https://
First install lispmob:
cd /home/user/lispmob-ovs/ make sudo make install sudo depmod -a
Afterward install the modified LISP-enabled OpenVswitch:
cd /home/user/ovs-lisp/ ./boot.sh ./configure --with-linux=/lib/modules/`uname -r`/build make
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/ sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
...and remove 'Module.symvers' from the 'clean' target (on line 33)
sudo reboot
Then, you will need to REMAKE and REINSTALL lisp
cd /home/lakafosi/lispmob-ovs make sudo make install sudo depmod -a cd /home/lakafosi/ovs-lisp/ make sudo make install sudo depmod -a sudo dmesg -c
In order to bring up the OVS bridge:
sudo /home/user/ovs-scripts/ovs-start
which includes:
#!/bin/bash BUILD_DIR=/home/lakafosi/ovs-lisp OVS_PATH=/usr/local rmmod -s openvswitch modprobe lisp insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko mkdir -p ${OVS_PATH}/etc/openvswitch rm ${OVS_PATH}/etc/openvswitch/conf.db ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema ${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ --remote=db:Open_vSwitch,manager_options \ --private-key=db:SSL,private_key \ --certificate=db:SSL,certificate \ --bootstrap-ca-cert=db:SSL,ca_cert \ --pidfile --detach ${OVS_PATH}/bin/ovs-vsctl --no-wait init ${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
You can verify everything is setup correctly by issuing:
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
which should yield an output similar to the following:
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server 1409 ? 00:00:00 ovs-vswitchd 1410 ? 00:00:00 ovs-vswitchd lisp 41286 1 openvswitch
Openstack
First create the stack user in order to install and run Openstack without any passwords requested.
su - useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers su stack cd
This Openstack installation is based on Devstack.
It is important you stick with the stable Folsom version:
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
Alternatively,
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny CirrOS cloud guest).
mkdir /home/stack/Downloads mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
All-In-One Openstack Node
$ su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
#!/bin/bash echo "=== lakafosi: Stopping firewall and allowing everyone... ===" iptables -F iptables -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X iptables -P INPUT ACCEPT iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT echo "=== lakafosi: Fixing the bridges... ===" VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}') #Alternatively: #VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i) echo "MAC address of guest VM: $VM_mac" brctl delif br100 vnet0 brctl delif br100 eth0 #ifconfig br100 0.0.0.0 #ifconfig br100 0.0.0.0 #ifconfig br100 192.168.127.1 netmask 255.255.255.0 ifconfig br100 down brctl delbr br100 ovs-vsctl -- --if-exists del-br br-lisp ovs-vsctl add-br br-lisp ovs-vsctl add-port br-lisp vnet0 #AllInOne #ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10 ovs-vsctl add-port br-lisp eth0 ovs-ofctl del-flows br-lisp #ovs-ofctl add-flows br-lisp /home/stack/Downloads/lisp_flow_table-AllInOne ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2 ovs-ofctl add-flow br-lisp priority=0,action=NORMAL ifconfig vnet0 up ifconfig eth0 0.0.0.0 #AllInOne ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192 route add default gw 10.10.1.1 killall farpd farpd -i br-lisp 192.168.127.0/24 echo "=== lakafosi: Bridge Status: ===" brctl show ovs-vsctl show ovs-appctl fdb/show br-lisp ovs-dpctl show ovs-ofctl dump-flows br-lisp echo "=== lakafosi: Go and START lisp...! ===" #echo "lakafosi: Starting lisp..." #lispd -f /etc/lispd.conf & echo "=== lakafosi: lisp status: ===" ps -ea | grep lisp; lsmod | grep lisp
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh /home/stack/Downloads/ovs-lisp-openstask-scenario.sh