DocWiki - User contributions [en]

Openstack with LISP-enabled OpenVSwitch

2013-03-20T23:17:03Z

Lakafosi: Moving to docwiki-dev

Openstack with LISP-enabled OpenVSwitch

2012-12-06T01:20:29Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| 10.10.10.10 | | 10.10.11.10 |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd root/src/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd root/src/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd root/src/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp root/src/lispmob-ovs/lisp_mod/Module.symvers root/src/ovs-lisp/datapath/linux/
sudo vi root/src/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp and reboot!
</p>
<pre>
cd root/src/lispmob-ovs
make
sudo make install
sudo depmod -a

cd root/src/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo reboot
</pre>

Please retry the above in case you get the following message:
"WARNING: "eid_locator4" [/root/src/ovs-lisp/datapath/linux/openvswitch.ko] undefined!"

<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo root/src/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=root/src/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db root/src/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq
1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact lakafosi@cisco.com.

Openstack with LISP-enabled OpenVSwitch

2012-12-06T01:19:02Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| 10.10.10.10 | | 10.10.11.10 |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd root/src/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd root/src/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd root/src/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp root/src/lispmob-ovs/lisp_mod/Module.symvers root/src/ovs-lisp/datapath/linux/
sudo vi root/src/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd root/src/lispmob-ovs
make
sudo make install
sudo depmod -a

cd root/src/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

Please try the above in case you get the following message:
"WARNING: "eid_locator4" [/root/src/ovs-lisp/datapath/linux/openvswitch.ko] undefined!"

<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo root/src/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=root/src/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db root/src/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq
1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact lakafosi@cisco.com.

Openstack with LISP-enabled OpenVSwitch

2012-12-06T01:17:46Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| 10.10.10.10 | | 10.10.11.10 |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd root/src/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd root/src/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd root/src/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp root/src/lispmob-ovs/lisp_mod/Module.symvers root/src/ovs-lisp/datapath/linux/
sudo vi root/src/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd root/src/lispmob-ovs
make
sudo make install
sudo depmod -a

cd root/src/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

Please try the above in case you get the following message:
"WARNING: "eid_locator4" [/root/src/ovs-lisp/datapath/linux/openvswitch.ko] undefined!"

<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo root/src/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=root/src/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db root/src/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact lakafosi@cisco.com.

Openstack with LISP-enabled OpenVSwitch

2012-12-06T00:40:39Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| 10.10.10.10 | | 10.10.11.10 |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/user/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/user/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

Please try the above in case you get the following message:
"WARNING: "eid_locator4" [/root/src/ovs-lisp/datapath/linux/openvswitch.ko] undefined!"

<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/user/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/user/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact lakafosi@cisco.com.

Openstack with LISP-enabled OpenVSwitch

2012-12-06T00:38:59Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| 10.10.10.10 | | 10.10.11.10 |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

Please try the above in case you get the following message:
"WARNING: "eid_locator4" [/root/src/ovs-lisp/datapath/linux/openvswitch.ko] undefined!"

<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact lakafosi@cisco.com.

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:15:06Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| 10.10.10.10 | | 10.10.11.10 |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact lakafosi@cisco.com.

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:13:37Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
10.10.10.10 10.10.11.10
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact lakafosi@cisco.com.

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:12:20Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact lakafosi@cisco.com.

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:11:43Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<h1> Demo video </h1>

For a copy of the video, please contact [[lakafosi@cisco.com]].

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:09:03Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:08:25Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:07:43Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|630px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[Image:Instances_-_OpenStack_Dashboard1.png|850px|middle]]

[[Image:Instance_Detail_-_OpenStack_Dashboard.png|850px|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:06:35Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|650px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|800px|frame|left]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|800px|frame|left]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:05:53Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[Image:OpenstackandLISP_Testbed.png|800px|middle]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|800px|frame|left]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|800px|frame|left]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:03:53Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>

or

[[File:OpenstackandLISP Testbed.png]]

[[Image:Instances_-_OpenStack_Dashboard1.png|800px|frame|left]]

[[Image:Instances_-_OpenStack_Dashboard1.png|400px|frame|left]]

[[Image:Instances_-_OpenStack_Dashboard1.png|400px|left]]

<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|800px|frame|left]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|800px|frame|left]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-05T19:01:44Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

or

[[File:OpenstackandLISP Testbed.png]]

[] OVS-LISP bridge
() Network interface
</pre>
<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|800px|frame|left]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|800px|frame|left]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

File:OpenstackandLISP Testbed.png

2012-12-05T18:57:54Z

Lakafosi:

Openstack with LISP-enabled OpenVSwitch

2012-12-05T18:56:56Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>

<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|800px|frame|left]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|800px|frame|left]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openwrt

2012-12-03T18:44:12Z

Lakafosi: Created page with "The following notes are based on x86 and AR71xx (on [http://wiki.openwrt.org/toh/linksys/wrt160nl Linksys WRT160NL]) architecture-based images of Attitude Adjustment and Backfire..."

The following notes are based on x86 and AR71xx (on [http://wiki.openwrt.org/toh/linksys/wrt160nl Linksys WRT160NL]) architecture-based images of Attitude Adjustment and Backfire versions of [https://openwrt.org/ Openwrt], but should hold for all Openwrt images.

= Debugging =

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:37:26Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>
<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>
<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|800px|frame|left]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|800px|frame|left]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:36:46Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>
<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>
<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|900px|frame|left]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|900px|thumb|left]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:35:42Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>
<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>
<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|900px|thumb|left]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|900px|thumb|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:34:58Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>
<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>
<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

[[File:Instances_-_OpenStack_Dashboard1.png|200px|thumb|middle]]

[[File:Instance_Detail_-_OpenStack_Dashboard.png|200px|thumb|middle]]

<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:32:31Z

Lakafosi:

<p>This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.
</p>
<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>
<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

<h3> Bringing up a guest VM </h3>

</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:28:41Z

Lakafosi: /* Bringing up a guest VM */

<p>This wiki describes all the steps necessary to setup a testbed of <a href="http://www.openstack.org/software/folsom/">Openstack Folsom</a> nodes (All-in-one node and Compute nodes), which rely on a <a href="http://lisp.cisco.com/">LISP-enabled</a> <a href="http://openvswitch.org/">OpenVSwitch (OVS)</a> bridge.
</p>
<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>
<p>__TOC__
</p><p><br />
</p>
<h1> Testbed Setup </h1>
<h2> Dependencies </h2>
<p>The Openstack nodes are considered hereafter to be <a href="http://releases.ubuntu.com/precise/">Ubuntu 12.04.1 LTS (Precise Pangolin)</a> machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.
</p>
<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>
<p>In case you wish to also use Wireshark (with LISP dissector included):
</p>
<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>
<p>Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
</p>
<ul><li> 1.8GB for the All-In-One Node
</li><li> 1.2GB for the Compute Node
</li></ul>
<p><br />
</p>
<h2> Network </h2>
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
<p>If instead of physical machines you plan on setting this testbed up with <b>VMware Fusion</b>, make sure the network is configured as <i>Bridged (Ethernet).</i>
</p><p><br />
</p>
<h2> Map Server (MS) </h2>
<p>Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.
</p><p>Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.
</p>
<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>
<h1> Installing the LISP-enabled OVS bridge </h1>
<p>On both machines do the following.
First download the corresponding code:
</p>
<pre>
cd /home/user/
git clone https://
</pre>
<p>First install lispmob:
</p>
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>
<p>Afterward install the modified LISP-enabled OpenVswitch:
</p>
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>
<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
<p>...and remove 'Module.symvers' from the 'clean' target (on line 33)
</p>
<pre>
sudo reboot
</pre>
<p>Then, you will need to REMAKE and REINSTALL lisp
</p>
<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>
<p>In order to bring up the OVS bridge:
</p>
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
<p>which includes:
</p>
<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>You can verify everything is setup correctly by issuing:
</p>
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
<p>which should yield an output similar to the following:
</p>
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>
<h1> Openstack </h1>
<p>First create the stack user in order to install and run Openstack without any passwords requested.
</p>
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>
<p>This Openstack installation is based on <a href="http://devstack.org/">Devstack</a>.
</p><p>It is important you stick with the stable Folsom version:
</p>
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
<p>Alternatively,
</p>
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>
<p>Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny <a href="https://launchpad.net/cirros/+download">CirrOS cloud guest</a>).
</p>
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>
<p><br />
</p>
<h2> All-In-One Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>This is the script run right after <i>stack.sh</i>, i.e. the Openstack installation. It is located in /home/stack/devstack/ .
</p>
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>
<p><br />
</p>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<h3> Bringing up a guest VM </h3>
<p><img src="/w/images/6/6e/Instances_-_OpenStack_Dashboard1.png" _fck_mw_filename="Instances - OpenStack Dashboard1.png" _fck_mw_width="900" _fck_mw_type="thumb" alt="Instances - OpenStack Dashboard1.png" class="fck_mw_frame fck_mw_right" />
<img src="/w/images/thumb/c/ca/Instance_Detail_-_OpenStack_Dashboard.png/900px-Instance_Detail_-_OpenStack_Dashboard.png" _fck_mw_filename="Instance Detail - OpenStack Dashboard.png" _fck_mw_width="900" _fck_mw_type="thumb" alt="Instance Detail - OpenStack Dashboard.png" class="fck_mw_frame fck_mw_right" />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<h2> Compute Openstack Node </h2>
<h3> localrc </h3>
<p>Located in /home/stack/devstack/
</p>
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>
<p><br />
</p>
<h3> local.sh </h3>
<p>Make sure no local.sh exists under /home/stack/devstack
</p>
<pre>
rm /home/stack/devstack/local.sh
</pre>
<h3> lispd.conf </h3>
<p>LISP is run with
</p>
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
<p>Its configuration file is located in /etc/ and contains:
</p>
<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>
<p><br />
</p>
<h3> ovs-lisp-openstask-scenario.sh </h3>
<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>
<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>
<p>To run it:
</p>
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:28:18Z

Lakafosi: /* Bringing up a guest VM */

This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.

<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>

__TOC__

= Testbed Setup =

== Dependencies ==

The Openstack nodes are considered hereafter to be [http://releases.ubuntu.com/precise/ Ubuntu 12.04.1 LTS (Precise Pangolin)] machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.

<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>

In case you wish to also use Wireshark (with LISP dissector included):

<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>

Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
* 1.8GB for the All-In-One Node
* 1.2GB for the Compute Node

== Network ==
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).''

== Map Server (MS) ==

Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.

Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.

<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>

= Installing the LISP-enabled OVS bridge =

On both machines do the following.
First download the corresponding code:

<pre>
cd /home/user/
git clone https://
</pre>

First install lispmob:
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>

Afterward install the modified LISP-enabled OpenVswitch:
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>

<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
...and remove 'Module.symvers' from the 'clean' target (on line 33)

<pre>
sudo reboot
</pre>

Then, you will need to REMAKE and REINSTALL lisp

<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

In order to bring up the OVS bridge:
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
which includes:

<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>

You can verify everything is setup correctly by issuing:
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
which should yield an output similar to the following:
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>

= Openstack =

First create the stack user in order to install and run Openstack without any passwords requested.
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>

This Openstack installation is based on [http://devstack.org/ Devstack].

It is important you stick with the stable Folsom version:
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
Alternatively,
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>

Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny [https://launchpad.net/cirros/+download CirrOS cloud guest]).
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>

== All-In-One Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
This is the script run right after ''stack.sh'', i.e. the Openstack installation. It is located in /home/stack/devstack/ .
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== Bringing up a guest VM ===

[[File:Instances_-_OpenStack_Dashboard1.png|900px|thumb|frame|middle]]
[[File:Instance_Detail_-_OpenStack_Dashboard.png|900px|thumb|middle]]

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

== Compute Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
Make sure no local.sh exists under /home/stack/devstack
<pre>
rm /home/stack/devstack/local.sh
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:28:06Z

Lakafosi: /* Bringing up a guest VM */

This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.

<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>

__TOC__

= Testbed Setup =

== Dependencies ==

The Openstack nodes are considered hereafter to be [http://releases.ubuntu.com/precise/ Ubuntu 12.04.1 LTS (Precise Pangolin)] machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.

<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>

In case you wish to also use Wireshark (with LISP dissector included):

<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>

Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
* 1.8GB for the All-In-One Node
* 1.2GB for the Compute Node

== Network ==
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).''

== Map Server (MS) ==

Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.

Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.

<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>

= Installing the LISP-enabled OVS bridge =

On both machines do the following.
First download the corresponding code:

<pre>
cd /home/user/
git clone https://
</pre>

First install lispmob:
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>

Afterward install the modified LISP-enabled OpenVswitch:
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>

<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
...and remove 'Module.symvers' from the 'clean' target (on line 33)

<pre>
sudo reboot
</pre>

Then, you will need to REMAKE and REINSTALL lisp

<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

In order to bring up the OVS bridge:
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
which includes:

<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>

You can verify everything is setup correctly by issuing:
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
which should yield an output similar to the following:
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>

= Openstack =

First create the stack user in order to install and run Openstack without any passwords requested.
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>

This Openstack installation is based on [http://devstack.org/ Devstack].

It is important you stick with the stable Folsom version:
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
Alternatively,
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>

Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny [https://launchpad.net/cirros/+download CirrOS cloud guest]).
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>

== All-In-One Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
This is the script run right after ''stack.sh'', i.e. the Openstack installation. It is located in /home/stack/devstack/ .
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== Bringing up a guest VM ===

[[File:Instances_-_OpenStack_Dashboard1.png|900px|thumb|middle]]
[[File:Instance_Detail_-_OpenStack_Dashboard.png|900px|thumb|middle]]

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

== Compute Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
Make sure no local.sh exists under /home/stack/devstack
<pre>
rm /home/stack/devstack/local.sh
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:27:39Z

Lakafosi: /* Bringing up a guest VM */

This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.

<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>

__TOC__

= Testbed Setup =

== Dependencies ==

The Openstack nodes are considered hereafter to be [http://releases.ubuntu.com/precise/ Ubuntu 12.04.1 LTS (Precise Pangolin)] machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.

<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>

In case you wish to also use Wireshark (with LISP dissector included):

<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>

Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
* 1.8GB for the All-In-One Node
* 1.2GB for the Compute Node

== Network ==
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).''

== Map Server (MS) ==

Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.

Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.

<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>

= Installing the LISP-enabled OVS bridge =

On both machines do the following.
First download the corresponding code:

<pre>
cd /home/user/
git clone https://
</pre>

First install lispmob:
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>

Afterward install the modified LISP-enabled OpenVswitch:
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>

<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
...and remove 'Module.symvers' from the 'clean' target (on line 33)

<pre>
sudo reboot
</pre>

Then, you will need to REMAKE and REINSTALL lisp

<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

In order to bring up the OVS bridge:
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
which includes:

<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>

You can verify everything is setup correctly by issuing:
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
which should yield an output similar to the following:
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>

= Openstack =

First create the stack user in order to install and run Openstack without any passwords requested.
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>

This Openstack installation is based on [http://devstack.org/ Devstack].

It is important you stick with the stable Folsom version:
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
Alternatively,
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>

Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny [https://launchpad.net/cirros/+download CirrOS cloud guest]).
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>

== All-In-One Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
This is the script run right after ''stack.sh'', i.e. the Openstack installation. It is located in /home/stack/devstack/ .
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== Bringing up a guest VM ===

[[File:Instances_-_OpenStack_Dashboard1.png|900px|frame|none]]
[[File:Instance_Detail_-_OpenStack_Dashboard.png|900px|frame|none]]

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

== Compute Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
Make sure no local.sh exists under /home/stack/devstack
<pre>
rm /home/stack/devstack/local.sh
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:27:07Z

Lakafosi: /* Bringing up a guest VM */

This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.

<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>

__TOC__

= Testbed Setup =

== Dependencies ==

The Openstack nodes are considered hereafter to be [http://releases.ubuntu.com/precise/ Ubuntu 12.04.1 LTS (Precise Pangolin)] machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.

<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>

In case you wish to also use Wireshark (with LISP dissector included):

<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>

Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
* 1.8GB for the All-In-One Node
* 1.2GB for the Compute Node

== Network ==
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).''

== Map Server (MS) ==

Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.

Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.

<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>

= Installing the LISP-enabled OVS bridge =

On both machines do the following.
First download the corresponding code:

<pre>
cd /home/user/
git clone https://
</pre>

First install lispmob:
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>

Afterward install the modified LISP-enabled OpenVswitch:
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>

<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
...and remove 'Module.symvers' from the 'clean' target (on line 33)

<pre>
sudo reboot
</pre>

Then, you will need to REMAKE and REINSTALL lisp

<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

In order to bring up the OVS bridge:
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
which includes:

<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>

You can verify everything is setup correctly by issuing:
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
which should yield an output similar to the following:
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>

= Openstack =

First create the stack user in order to install and run Openstack without any passwords requested.
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>

This Openstack installation is based on [http://devstack.org/ Devstack].

It is important you stick with the stable Folsom version:
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
Alternatively,
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>

Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny [https://launchpad.net/cirros/+download CirrOS cloud guest]).
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>

== All-In-One Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
This is the script run right after ''stack.sh'', i.e. the Openstack installation. It is located in /home/stack/devstack/ .
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== Bringing up a guest VM ===

[[Image:Instances_-_OpenStack_Dashboard1.png|600px|frame|none]]
[[Image:Instance_Detail_-_OpenStack_Dashboard.png|600px|frame|none]]

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

== Compute Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
Make sure no local.sh exists under /home/stack/devstack
<pre>
rm /home/stack/devstack/local.sh
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:26:47Z

Lakafosi: /* Bringing up a guest VM */

This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.

<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>

__TOC__

= Testbed Setup =

== Dependencies ==

The Openstack nodes are considered hereafter to be [http://releases.ubuntu.com/precise/ Ubuntu 12.04.1 LTS (Precise Pangolin)] machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.

<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>

In case you wish to also use Wireshark (with LISP dissector included):

<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>

Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
* 1.8GB for the All-In-One Node
* 1.2GB for the Compute Node

== Network ==
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).''

== Map Server (MS) ==

Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.

Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.

<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>

= Installing the LISP-enabled OVS bridge =

On both machines do the following.
First download the corresponding code:

<pre>
cd /home/user/
git clone https://
</pre>

First install lispmob:
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>

Afterward install the modified LISP-enabled OpenVswitch:
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>

<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
...and remove 'Module.symvers' from the 'clean' target (on line 33)

<pre>
sudo reboot
</pre>

Then, you will need to REMAKE and REINSTALL lisp

<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

In order to bring up the OVS bridge:
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
which includes:

<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>

You can verify everything is setup correctly by issuing:
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
which should yield an output similar to the following:
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>

= Openstack =

First create the stack user in order to install and run Openstack without any passwords requested.
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>

This Openstack installation is based on [http://devstack.org/ Devstack].

It is important you stick with the stable Folsom version:
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
Alternatively,
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>

Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny [https://launchpad.net/cirros/+download CirrOS cloud guest]).
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>

== All-In-One Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
This is the script run right after ''stack.sh'', i.e. the Openstack installation. It is located in /home/stack/devstack/ .
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== Bringing up a guest VM ===

[[Image:Instances_-_OpenStack_Dashboard1.png|900px|frame|none]]
[[Image:Instance_Detail_-_OpenStack_Dashboard.png|900px|frame|none]]

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

== Compute Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
Make sure no local.sh exists under /home/stack/devstack
<pre>
rm /home/stack/devstack/local.sh
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:26:08Z

Lakafosi: /* Bringing up a guest VM */

This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.

<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>

__TOC__

= Testbed Setup =

== Dependencies ==

The Openstack nodes are considered hereafter to be [http://releases.ubuntu.com/precise/ Ubuntu 12.04.1 LTS (Precise Pangolin)] machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.

<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>

In case you wish to also use Wireshark (with LISP dissector included):

<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>

Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
* 1.8GB for the All-In-One Node
* 1.2GB for the Compute Node

== Network ==
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).''

== Map Server (MS) ==

Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.

Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.

<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>

= Installing the LISP-enabled OVS bridge =

On both machines do the following.
First download the corresponding code:

<pre>
cd /home/user/
git clone https://
</pre>

First install lispmob:
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>

Afterward install the modified LISP-enabled OpenVswitch:
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>

<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
...and remove 'Module.symvers' from the 'clean' target (on line 33)

<pre>
sudo reboot
</pre>

Then, you will need to REMAKE and REINSTALL lisp

<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

In order to bring up the OVS bridge:
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
which includes:

<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>

You can verify everything is setup correctly by issuing:
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
which should yield an output similar to the following:
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>

= Openstack =

First create the stack user in order to install and run Openstack without any passwords requested.
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>

This Openstack installation is based on [http://devstack.org/ Devstack].

It is important you stick with the stable Folsom version:
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
Alternatively,
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>

Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny [https://launchpad.net/cirros/+download CirrOS cloud guest]).
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>

== All-In-One Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
This is the script run right after ''stack.sh'', i.e. the Openstack installation. It is located in /home/stack/devstack/ .
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== Bringing up a guest VM ===

[[Image:Instances_-_OpenStack_Dashboard1.png|900px|thumb|middle]]
[[Image:Instance_Detail_-_OpenStack_Dashboard.png|900px|thumb|middle]]

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

== Compute Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
Make sure no local.sh exists under /home/stack/devstack
<pre>
rm /home/stack/devstack/local.sh
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:24:39Z

Lakafosi: /* Bringing up a guest VM */

This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.

<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>

__TOC__

= Testbed Setup =

== Dependencies ==

The Openstack nodes are considered hereafter to be [http://releases.ubuntu.com/precise/ Ubuntu 12.04.1 LTS (Precise Pangolin)] machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.

<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>

In case you wish to also use Wireshark (with LISP dissector included):

<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>

Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
* 1.8GB for the All-In-One Node
* 1.2GB for the Compute Node

== Network ==
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).''

== Map Server (MS) ==

Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.

Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.

<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>

= Installing the LISP-enabled OVS bridge =

On both machines do the following.
First download the corresponding code:

<pre>
cd /home/user/
git clone https://
</pre>

First install lispmob:
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>

Afterward install the modified LISP-enabled OpenVswitch:
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>

<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
...and remove 'Module.symvers' from the 'clean' target (on line 33)

<pre>
sudo reboot
</pre>

Then, you will need to REMAKE and REINSTALL lisp

<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

In order to bring up the OVS bridge:
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
which includes:

<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>

You can verify everything is setup correctly by issuing:
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
which should yield an output similar to the following:
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>

= Openstack =

First create the stack user in order to install and run Openstack without any passwords requested.
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>

This Openstack installation is based on [http://devstack.org/ Devstack].

It is important you stick with the stable Folsom version:
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
Alternatively,
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>

Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny [https://launchpad.net/cirros/+download CirrOS cloud guest]).
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>

== All-In-One Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
This is the script run right after ''stack.sh'', i.e. the Openstack installation. It is located in /home/stack/devstack/ .
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== Bringing up a guest VM ===

[[Image:Instances_-_OpenStack_Dashboard1.png|frame|none]]
[[File:Instance_Detail_-_OpenStack_Dashboard.png|900px|thumb|left|alt text]]

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

== Compute Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
Make sure no local.sh exists under /home/stack/devstack
<pre>
rm /home/stack/devstack/local.sh
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

File:Instances - OpenStack Dashboard1.png

2012-12-02T12:22:24Z

Lakafosi:

File:Instance Detail - OpenStack Dashboard.png

2012-12-02T12:21:46Z

Lakafosi:

Openstack with LISP-enabled OpenVSwitch

2012-12-02T12:20:06Z

Lakafosi: /* All-In-One Openstack Node */

This wiki describes all the steps necessary to setup a testbed of [http://www.openstack.org/software/folsom/ Openstack Folsom] nodes (All-in-one node and Compute nodes), which rely on a [http://lisp.cisco.com/ LISP-enabled] [http://openvswitch.org/ OpenVSwitch (OVS)] bridge.

<pre>
cisco ("public") Network
----+----------------------+-----
| |
+--------+--------+ +-------+--------+
| All-In-One Node | | Compute Node |
| | | |
| | | |
| Nova | | Nova -agent |
| Keystone | | Nova-compute |
| Glance | | Nova-Network |
| Nova-Network | +----------------+
| Cinder |
| Horizon |
+-----------------+
</pre>

__TOC__

= Testbed Setup =

== Dependencies ==

The Openstack nodes are considered hereafter to be [http://releases.ubuntu.com/precise/ Ubuntu 12.04.1 LTS (Precise Pangolin)] machines (running the 3.2.0-29 kernel version).
The following dependencies need to be met.

<pre>
sudo apt-get update
sudo apt-get upgrade (NOT dist-upgrade)

sudo apt-get install openssh-server vim git openssh-server bridge-utils build-essential git-core libssl-dev libconfuse-dev pkg-config autoconf libtool pkg-config gengetopt \
clang mosh cscope vim-gtk htop subversion tmux iotop dpatch automake1.9 python-support uml-utilities farpd

sudo apt-get install linux-headers-3.2.0-29-generic

# For KVM
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager virtinst
</pre>

In case you wish to also use Wireshark (with LISP dissector included):

<pre>
$ sudo apt-get build-dep wireshark
$ sudo apt-get install autoconf bison flex libtool libgtk2.0-dev libpcap-dev libc-ares-dev libsmi2-dev libgnutls-dev libgcrypt11-dev libkrb5-dev libcap2-bin libgeoip-dev libortp-dev libportaudio-dev

$ mkdir ~/Downloads; cd ~/Downloads; wget http://wiresharkdownloads.riverbed.com/wireshark/src/wireshark-1.8.3.tar.bz2
$ tar -xvf wireshark-1.8.3.tar.bz2 -C /tmp
$ cd /tmp/wireshark-1.8.3/
$ ./autogen.sh
$ ./configure --enable-setcap-install
$ make
$ sudo make install
$ sudo ldconfig
$ wireshark &
</pre>

Although this is simply a recommendation, the testbed has been successfully run with the following memory settings:
* 1.8GB for the All-In-One Node
* 1.2GB for the Compute Node

== Network ==
<pre>
Public + Management Network
--+--------------------------+-----
| |
| |
10.10.10.10 10.10.11.10
| |
+------|-------------+ +--------|-------+
| (eth0) | | (eth0) |
| | | | | |
| | | | | |
| [ br-lisp ] | | [br-lisp] |
| | | | | | |
| (vnet0) (vnet1) | | (vnet0) |
| | | | | | |
| VM VM | | VM |
| 192.168. 192.168. | | 192.168. |
| 127.2 127.4 | | 127.3 |
+--------------------+ +----------------+
All-In-One Node Compute Node

[] OVS-LISP bridge
() Network interface
</pre>
If instead of physical machines you plan on setting this testbed up with '''VMware Fusion''', make sure the network is configured as ''Bridged (Ethernet).''

== Map Server (MS) ==

Just to get an idea of how the EID addresses are supposed to be associated with their corresponding RLOCs, you can have a look at the following MS configuration, which makes static assignments.

Under regular LISP operation you will not have to impose the following configuration. Rather, it is formed automatically and dynamically.

<pre>
<?xml version="1.0" encoding="ISO-8859-1"?>

<db>

<eid prefix="0.0.0.0/3" act="2" ttl="5" />
<eid prefix="128.0.0.0/8" act="2" ttl="5" />
<eid prefix="171.0.0.0/8" act="2" ttl="5" />
<eid prefix="224.0.0.0/4" act="2" ttl="5" />

<eid prefix="192.168.127.3/32" ttl="1440" A="true">
<rloc>
<address>10.10.10.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

<eid prefix="192.168.127.4/32" ttl="1440" A="true">
<rloc>
<address>10.10.11.10</address>
<priority>1</priority>
<weight>100</weight>
<m_priority>255</m_priority>
<m_weight>0</m_weight>
<reachable>true</reachable>
</rloc>
</eid>

</db>
</pre>

= Installing the LISP-enabled OVS bridge =

On both machines do the following.
First download the corresponding code:

<pre>
cd /home/user/
git clone https://
</pre>

First install lispmob:
<pre>
cd /home/user/lispmob-ovs/
make
sudo make install
sudo depmod -a
</pre>

Afterward install the modified LISP-enabled OpenVswitch:
<pre>
cd /home/user/ovs-lisp/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
</pre>

<pre>
cp /home/user/lispmob-ovs/lisp_mod/Module.symvers /home/user/ovs-lisp/datapath/linux/
sudo vi /home/user/ovs-lisp/datapath/linux/Makefile.main.in
</pre>
...and remove 'Module.symvers' from the 'clean' target (on line 33)

<pre>
sudo reboot
</pre>

Then, you will need to REMAKE and REINSTALL lisp

<pre>
cd /home/lakafosi/lispmob-ovs
make
sudo make install
sudo depmod -a

cd /home/lakafosi/ovs-lisp/
make
sudo make install
sudo depmod -a

sudo dmesg -c
</pre>

In order to bring up the OVS bridge:
<pre>
sudo /home/user/ovs-scripts/ovs-start
</pre>
which includes:

<pre>
#!/bin/bash
BUILD_DIR=/home/lakafosi/ovs-lisp
OVS_PATH=/usr/local

rmmod -s openvswitch
modprobe lisp
insmod ${BUILD_DIR}/datapath/linux/openvswitch.ko
mkdir -p ${OVS_PATH}/etc/openvswitch
rm ${OVS_PATH}/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db /home/lakafosi/ovs-lisp/vswitchd/vswitch.ovsschema
${OVS_PATH}/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--private-key=db:SSL,private_key \
--certificate=db:SSL,certificate \
--bootstrap-ca-cert=db:SSL,ca_cert \
--pidfile --detach
${OVS_PATH}/bin/ovs-vsctl --no-wait init
${OVS_PATH}/sbin/ovs-vswitchd --pidfile --detach
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>

You can verify everything is setup correctly by issuing:
<pre>
ps -ea | grep ovs; ps -ea | grep lisp; lsmod | grep lisp; lsmod | grep ovs
</pre>
which should yield an output similar to the following:
<pre>
1404 ? 00:00:00 ovs_workq 1406 ? 00:00:00 ovsdb-server
1409 ? 00:00:00 ovs-vswitchd
1410 ? 00:00:00 ovs-vswitchd
lisp 41286 1 openvswitch
</pre>

= Openstack =

First create the stack user in order to install and run Openstack without any passwords requested.
<pre>
su -
useradd -U -G sudo -s /bin/bash -m stack; echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su stack
cd
</pre>

This Openstack installation is based on [http://devstack.org/ Devstack].

It is important you stick with the stable Folsom version:
<pre>
git clone -b stable/folsom https://github.com/openstack-dev/devstack.git
</pre>
Alternatively,
<pre>
git clone -b stable/folsom git://git.cisco.com/lakafosi/devstack.git
</pre>

Additionally, you will have to download one more script for setting up the OVS-bridge and attaching the vnetX ports of the instantiated VMs, the lispd.conf control plane configuration file of LISP and a few cloud qcow2 (Ubuntu and Fedora) images (to use in addition to the preloaded tiny [https://launchpad.net/cirros/+download CirrOS cloud guest]).
<pre>
mkdir /home/stack/Downloads
mv /* /home/stack/devstack/Downloads/* /home/stack/Downloads
</pre>

== All-In-One Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#HOST_NAME=$(hostname)
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.10.10

FIXED_RANGE=192.168.127.0/24

MULTI_HOST=True
SCHEDULER=nova.scheduler.simple.SimpleScheduler

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
This is the script run right after ''stack.sh'', i.e. the Openstack installation. It is located in /home/stack/devstack/ .
<pre>
#!/usr/bin/env bash

# Sample ``local.sh`` for user-configurable tasks to run automatically
# at the sucessful conclusion of ``stack.sh``.

# Keep track of the devstack directory
TOP_DIR=$(cd $(dirname "$0") && pwd)
# Import common functions
source $TOP_DIR/functions
# Use openrc + stackrc + localrc for settings
source $TOP_DIR/stackrc
# Destination path for installation ``DEST``
DEST=${DEST:-/opt/stack}

# lakafosi
for i in `seq 2 5`; do /opt/stack/nova/bin/nova-manage fixed reserve 192.168.127.$i; done

# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2
# nova-manage floating create --ip_range=192.168.127.6/31

# Get OpenStack demo auth
source $TOP_DIR/openrc demo demo
glance add name=cirros-0.3.0-x86_64 disk_format=qcow2 container_format=bare < ~/Downloads/cirros-0.3.0-x86_64-disk.img
glance add name=Ubuntu-12.04 is_public=true container_format=ovf disk_format=qcow2 < ~/Downloads/precise-server-cloudimg-amd64-disk1.img
glance add name=f16-jeos is_public=true disk_format=qcow2 container_format=bare < ~/Downloads/f16-x86_64-openstack-sda.qcow2

# Import keys from the current user into the default OpenStack user (usually
# ``demo``)

# Get OpenStack auth
source $TOP_DIR/openrc
# Add first keypair found in localhost:$HOME/.ssh
for i in $HOME/.ssh/id_rsa.pub $HOME/.ssh/id_dsa.pub; do
if [[ -r $i ]]; then
nova keypair-add --pub_key=$i `hostname`
break
fi
done

# Create my Flavor
# ---------------
# Get OpenStack admin auth
source $TOP_DIR/openrc admin admin
# Name of new flavor
# set in ``localrc`` with ``DEFAULT_INSTANCE_TYPE=m1.micro``
MI_NAME=m1.cirrOS
# Create micro flavor if not present
if [[ -z $(nova flavor-list | grep $MI_NAME) ]]; then
nova-manage instance_type create m1.cirrOS 256 1 0 0 0 0
# nova flavor-create $MI_NAME 6 128 0 1
fi

# Other Uses
# ----------

# Add tcp/22 and icmp to default security group
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.3/32 # for Openstack AllInOne Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== Bringing up a guest VM ===

[[Image:901CVPSIPDeployment.png|frame|none]]

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#AllInOne
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.11.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#AllInOne
ifconfig br-lisp 10.10.10.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

== Compute Openstack Node ==

=== localrc ===
Located in /home/stack/devstack/
<pre>
#SERVICE_HOST_NAME=lakafosi-server-1
SERVICE_HOST=10.10.10.10
HOST_IP=10.10.11.10

MULTI_HOST=True

#FIXED_RANGE=192.168.127.0/24

# compute node
ENABLED_SERVICES=n-cpu,n-net,n-api,n-vol,rabbit
#disable_all_services
#enable_service rabbit n-cpu quantum q-agt n-vol n-api

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

CINDER_BRANCH=stable/folsom
NOVA_BRANCH=stable/folsom
SWIFT_BRANCH=stable/folsom
GLANCE_BRANCH=stable/folsom
KEYSTONE_BRANCH=stable/folsom
HORIZON_BRANCH=stable/folsom
#QUANTUM_BRANCH=stable/folsom

# password
MYSQL_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment the following line to save time by skipping checking of packages that need to be installed
#OFFLINE=True
</pre>

=== local.sh ===
Make sure no local.sh exists under /home/stack/devstack
<pre>
rm /home/stack/devstack/local.sh
</pre>

=== lispd.conf ===
LISP is run with
<pre>
sudo lispd -f /etc/lispd.conf
</pre>
Its configuration file is located in /etc/ and contains:

<pre>
# lispd example config file

debug = on
map-request-retries = 2 # send 2 before giving up

# Encapsulated Map-Requests are sent to this map resolver
map-resolver = 10.32.164.141

# Register to this map server
map-server {
address = 10.32.164.141
key-type = 1 # SHA1
key = foo%bar
verify = off # on --> lig(self)
proxy-reply = on # ask ms to proxy reply
}

# Encapsulate packets for non-LISP sites to this Proxy-ETR
proxy-etr {
address = 149.20.48.60
priority = 1
weight = 100
}

# IPv4 EID of the mobile node
database-mapping {
eid-prefix = 192.168.127.4/32 # for Openstack Compute Node
# iid = 0
interface = br-lisp
priority = 1
weight = 100
}

# List of PITRs to SMR on handover
proxy-itrs = {
69.31.31.98, # eqx-ash-pxtr
149.20.48.60, # isc-pxtr
198.6.255.37, # asp-pxtr
129.250.1.63, # ntt-amer-pxtr
217.8.98.33, # intouch-pxtr-1
217.8.98.35, # intouch-pxtr-2
193.162.145.46, # tdc-pxtr
158.38.1.92, # uninett-pxtr
203.181.249.172 # apan-pxtr
}
</pre>

=== ovs-lisp-openstask-scenario.sh ===

<pre>
su -
vi /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>

<pre>
#!/bin/bash

echo "=== lakafosi: Stopping firewall and allowing everyone... ==="
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
echo "=== lakafosi: Fixing the bridges... ==="
VM_mac=fa:16:$(ifconfig vnet0 | grep HWaddr | awk '{print $5}'i | awk '{print substr($0,7)}')
#Alternatively:
#VM_mac=$(brctl showmacs br100 | grep fa:16: |awk '{print $2}'i)
echo "MAC address of guest VM: $VM_mac"
brctl delif br100 vnet0
brctl delif br100 eth0
#ifconfig br100 0.0.0.0
#ifconfig br100 0.0.0.0
#ifconfig br100 192.168.127.1 netmask 255.255.255.0
ifconfig br100 down
brctl delbr br100
ovs-vsctl -- --if-exists del-br br-lisp
ovs-vsctl add-br br-lisp
ovs-vsctl add-port br-lisp vnet0
#Compute
ovs-vsctl add-port br-lisp lisp0 -- set Interface lisp0 type=lisp options:remote_ip=10.10.10.10
ovs-vsctl add-port br-lisp eth0
ovs-ofctl del-flows br-lisp
ovs-ofctl add-flow br-lisp priority=3,dl_dst=00:11:22:ee:ee:ee,action=mod_dl_dst:$VM_mac,NORMAL
ovs-ofctl add-flow br-lisp priority=2,in_port=1,dl_type=0x0806,action=NORMAL
ovs-ofctl add-flow br-lisp priority=1,in_port=1,dl_type=0x0800,vlan_tci=0,nw_src=192.168.127.0/24,action=output:2
ovs-ofctl add-flow br-lisp priority=0,action=NORMAL
ifconfig vnet0 up
ifconfig eth0 0.0.0.0
#Compute
ifconfig br-lisp 10.10.11.10 netmask 255.255.255.192
route add default gw 10.10.1.1
killall farpd
farpd -i br-lisp 192.168.127.0/24
echo "=== lakafosi: Bridge Status: ==="
brctl show
ovs-vsctl show
ovs-appctl fdb/show br-lisp
ovs-dpctl show
ovs-ofctl dump-flows br-lisp
echo "=== lakafosi: Go and START lisp...! ==="
#echo "lakafosi: Starting lisp..."
#lispd -f /etc/lispd.conf &
echo "=== lakafosi: lisp status: ==="
ps -ea | grep lisp; lsmod | grep lisp
</pre>

To run it:
<pre>
chmod +x /home/stack/Downloads/ovs-lisp-openstask-scenario.sh
/home/stack/Downloads/ovs-lisp-openstask-scenario.sh
</pre>