OpenShift Origin Heat Deployment Guide

From DocWiki

(Difference between revisions)
Jump to: navigation, search
m (OpenShift Deployment Verification)
m (OpenShift Deployment Verification)
Line 313: Line 313:
| id                                  | stack_name | stack_status      | creation_time        |
| id                                  | stack_name | stack_status      | creation_time        |
| 349e7128-cd15-4333-91ed-dff4b9589a46 | oso-stack       | CREATE_COMPLETE    | 2014-03-07T18:41:07Z |
| 349e7128-cd15-4333-91ed-dff4b9589a46 | oso-stack | CREATE_COMPLETE    | 2014-03-07T18:41:07Z |

Revision as of 22:52, 7 March 2014



This document provides users step-by-step instructions for deploying OpenShift Origin (OSO) v3.0 using OpenStack Heat. Heat is the main project in the OpenStack Orchestration program. It implements an orchestration engine to launch multiple composite cloud applications based on templates in the form of text files that can be treated like code. The current OSO Heat template supports deploying one Broker instance and one Node instance. The base OpenStack deployment was accomplished using Cisco's OpenStack Installer (COI) Havana release. However, the Heat deployment should work with any OpenStack distribution that uses Icehouse-2 or later and Neutron VLAN Provider Networking. Therefore, if you are using COI to manage your OpenStack deployment, you must use either the full_ha or compressed_ha scenario. This is because both scenarios use Neutron VLAN Provider Networking. Please file a feature request if you require support for a different Neutron networking model.

Prepare the Images

Images must be created for the Broker and Node instance. The images will be used to deploy running Broker and Node instances using the OpenStack Image Service (Glance), Compute Service (Nova) and Orchestration Service (Heat).

Log into a host that contains the following:

  • OpenStack client packages (i.e. python-novaclient)
  • Network connectivity to OpenStack API endpoints
  • OpenStack credential file. (i.e. openrc). Here is a reference to the contents of an authentication file.

Install git and download Heat templates:

apt-get install -y git
git clone

Perform the following steps in the parent directory of heat-templates to build the images using diskimage-builder:

Download the diskimage-builder software and dependencies:

git clone
apt-get install -y qemu-utils kpartx policycoreutils

Create environmental variables for the Broker image:

mkdir $HOME/tmp
export DIB_RELEASE=19
export ELEMENTS_PATH=heat-templates/openshift-origin/F19/elements
export TMP_DIR=$HOME/tmp

Create the Broker image. Be patient as it takes several minutes to build the image:

diskimage-builder/bin/disk-image-create --no-tmpfs -a amd64 vm fedora openshift-origin-broker -o F19-x86_64-openshift-origin-broker

If you have not done so already, load your credential file.

source /root/openrc

Note: A credential file can be avoided by using the necessary Glance flags to specify the auth URL, username, password, etc..

Add the newly created Broker image to Glance:

glance image-create --name F19-x86_64-openshift-origin-broker --is-public true --disk-format qcow2 --container-format bare < F19-x86_64-openshift-origin-broker.qcow2

Change the DIB_IMAGE_SIZE environmental variable for the Node image:

export DIB_IMAGE_SIZE=20

Create the Node image. Be patient as it may take several minutes to build the image:

diskimage-builder/bin/disk-image-create --no-tmpfs -a amd64 vm fedora openshift-origin-node -o F19-x86_64-openshift-origin-node

Add the newly created Node image to Glance:

glance image-create --name F19-x86_64-openshift-origin-node --is-public true --disk-format qcow2 --container-format bare < F19-x86_64-openshift-origin-node.qcow2

Verify the Broker and Node images have been installed on Glance and has an active status:

glance image-list
| ID                                   | Name                                     | Disk Format | Container Format | Size       | Status |
| 102d07df-53e9-4499-b04d-32d9e217e2d1 | F19-x86_64-openshift-origin-broker       | qcow2       | bare             | 457444864  | active |
| 5a93c647-98c7-4b46-8461-ac17d3a1aee7 | F19-x86_64-openshift-origin-node         | qcow2       | bare             | 454047232  | active |

Follow the instructions in the SSH Key Injection Section of the Cisco Havana HA Manual Deployment Guide to create a Nova key-pair.

If Neutron networks have yet to be created, create your first tenant network now. Keep in mind this example uses Neutron Provider VLAN Networks which is the only supported networking model for the OSO Heat template. In our example, we use the admin tenant.  Create additional networks as needed. Note: The --tenant_id flag is not specified in the following commands because we previously sourced our credential file.

neutron net-create public223 --provider:network_type vlan --provider:physical_network physnet1 --provider:segmentation_id 223

Create your first tenant subnet and associate it to the network you created in the previous step. The example below uses .10-.250 for Instance IP addresses. Modify the allocation-pool and dns_nameservers based on your deployment needs. Create additional networks as needed.

neutron subnet-create --name 223-subnet --allocation-pool start=,end= public223 --dns_nameservers list=true

Configure the OpenStack DHCP Domain

Nova uses metadata to manage the hostname of instances. By default the hostname of instances will be the name of the instance in the Nova boot command, followed by a period and the domain. The domain is either novalocal or openstacklocal by default, depending on the method used for accessing metadata. This domain MUST match the domain used within your OpenShift deployment. The example below sets the domain to in /etc/nova/nova.conf:

vi /etc/nova/nova.conf

Restart the Nova API service:

service nova-api restart

The example below sets the domain to in /etc/neutron/dhcp_agent.ini

vi /etc/neutron/dhcp_agent.ini

Restart the Neutron DHCP Agent:

service neutron-dhcp-agent restart

Heat Deployment

Since the Icehouse release (2014.1.b2 or later) of Heat is required for the OSO template, Heat must be installed from source. At the time of this writing, the latest version of COI is Havana. If you have an existing Heat installation prior to 2014.1.b2, uninstall all Heat packages:

apt-get remove -y heat-common heat-api heat-api-cfn heat-api-cloudwatch heat-engine

If you are planning to deploy OpenStack using COI or you have just uninstalled the Heat packages from an existing COI deployment, remove Heat from the list of services that are managed by COI:

Comment-out heat_all from /etc/puppet/data/class_groups/controller.yaml


Install Heat from source:

apt-get install -y git
git clone
cd heat
git checkout 2014.1.b1

Install Heat source dependencies:

apt-get install python-dev python-pip libxml2-dev libxslt1-dev libsasl2-dev libsqlite3-dev libssl-dev libldap2-dev libffi-dev

Install Heat:

pip install -r ~/heat/test-requirements.txt
python ~/heat/ install

Create the Heat directories if they do not exist:

mkdir /etc/heat
mkdir /var/log/heat
chown heat:heat /etc/heat
chown heat:heat /var/log/heat

Copy the Heat config files:

cp -R ~/heat/etc/heat/* /etc/heat/.

Edit the /etc/heat/heat.conf file to include the following. Replace <CONTROLLER_IP> with the IP address of the Controller. Replace <CONTROLLER_VIP> with the VIP address of the Controller Cluster. Note: The example configuration below is based on the COI full_ha scenario. If you are not using the full_ha or compressed_ha scenario, use <CONTROLLER_IP> instead of <CONTROLLER_VIP> and follow the inline notes:

# Comment-out rabbit_hosts if not using COI full_ha scenario
# Comment-out rabbit_ha_queues if not using COI full_ha scenario

connection = mysql://heat:heat@<CONTROLLER_VIP>/heat
# Comment-out idle_timeout if not using COI full_ha scenario






Create the Heat database and necessary privileges:

mysql -u root -p
mysql> CREATE DATABASE heat;
mysql> GRANT ALL PRIVILEGES ON heat.* TO 'heat'@'localhost' DENTIFIED BY 'heat';
mysql> GRANT ALL PRIVILEGES ON heat.* TO 'heat'@'%' IDENTIFIED BY 'heat';

Start the Heat services:

/usr/bin/python /usr/local/bin/heat-api --config-file=/etc/heat/heat.conf &
/usr/bin/python /usr/local/bin/heat-api-cfn --config-file=/etc/heat/heat.conf &
/usr/bin/python /usr/local/bin/heat-api-cloudwatch --config-file=/etc/heat/heat.conf &
/usr/bin/python /usr/local/bin/heat-engine --config-file=/etc/heat/heat.conf &

Verify the Heat services are running:

 ps -ef | grep heat

OpenShift Deployment

At this point, you should have a functioning OpenStack deployment that includes Heat (2014.1.b2 or later) and that your OpenShift Broker and Node images have been successfully uploaded to Glance. Now simply issue the heat stack-create command to deploy your OpenShift Origin environment:

heat stack-create <STACK_NAME --template-file=<HEAT_TEMPLATE_FILE_PATH> \

Here is an overview of the parameters contained within the heat stack-create command:

  • <STACK_NAME>: The name of the Heat stack.
  • <HEAT_TEMPLATE_FILE_PATH>: The file path to the Heat OpenShift.template. If you are following these directions, the path should be: /root/heat-templates/openshift-origin/F19/OpenShift.template.
  • <KEY_NAME>: The name of the Nova key-pair created earlier in this document. (i.e. admin-key)
  • <DOMAIN_NAME>: The domain name that will be used by your OpenShift deployment. Defaults to
  • <UPSTREAM_DNS>: The IP address(es) of the upstream DNS serves used by your Broker instance for name resolution (outside of <DOMAIN_NAME>). Defaults to Google DNS (
  • <UPSTREAM_NTP>: IP address of the upstream NTP server. Defaults to ' iburst'. Note: iburst must be included in your NTP server entry and therefore a ' ' is required for this parameter.
  • <NEUTRON_NET_ID>: The ID of the Neutron network that will be used to spawn Broker and Node instances on. You can obtain the Neutron Net ID from the neutron net-list command.
  • <NEUTRON_SUBNET_ID>: The ID of the Neutron subnet that will be used to spawn Broker and Node instances on. You can obtain the Neutron Subnet ID from the neutron subnet-list command.
  • <BROKER_NAME>: The name that will be appended to <DOMAIN_NAME> and used as the hostname of the Broker instance. Defaults to openshift.brokerinstance.novalocal.
  • <NODE_NAME>: The name that will be appended to <DOMAIN_NAME> and used as the hostname of the Node instance. Defaults to openshift.nodeinstance.novalocal.
  • <USERNAME>: The username of the admin account for managing the OpenShift environment. Defaults to openshift.
  • <PASSWORD>: The password of the admin account for managing the OpenShift environment. Defaults to password.

Here is an example of the heat stack-create command with the required parameters supplied:

heat stack-create oso-stack --template-file=/root/heat-templates/openshift-origin/F19/OpenShift.template \
UpstreamNTP=' iburst';NetID=df4a1975-aa4f-40f5-bd86-f1a10ccbde46;\

OpenShift Deployment Verification

The OpenShift Heat stack takes anywhere from 20-30 minutes to be deployed and fully configured, so please be patient. You can observe the status of the deployment with the following steps:

Verify the progress of the OpenShift stack:

heat stack-list
| id                                   | stack_name | stack_status       | creation_time        |
| 7c366d13-08ec-41b1-aa3d-6f9e5a197140 | oso-stack  | CREATE_IN_PROGRESS | 2014-02-12T22:37:32Z |

Review the details of the OpenShift stack:

 heat stack-show <STACK_NAME>
| Property             | Value                                                                                                                      |
| capabilities         | []                                                                                                                         |
| creation_time        | 2014-03-07T18:41:07Z                                                                                                       |
| description          | Template for setting up an OpenShift Origin environment                                                                    |
| disable_rollback     | True                                                                                                                       |
| id                   | 349e7128-cd15-4333-91ed-dff4b9589a46                                                                                       |
| links                |             |
| notification_topics  | []                                                                                                                         |
| outputs              | [                                                                                                                          |
|                      |   {                                                                                                                        |
|                      |     "output_value": "nameserver",                                                                           |
|                      |     "description": "Entry to insert into /etc/resolv.conf for application host names to resolve",                          |
|                      |     "output_key": "NameServerEntry"                                                                                        |
|                      |   },                                                                                                                       |
|                      |   {                                                                                                                        |
|                      |     "output_value": "",                                                                      |
|                      |     "description": "URL for OpenShift Origin cConsole",                                                                    |
|                      |     "output_key": "OpenShiftConsole"                                                                                       |
|                      |   }                                                                                                                        |
|                      | ]                                                                                                                          |
| parameters           | {                                                                                                                          |
|                      |   "Username": "openshift",                                                                                                 |
|                      |   "BrokerFlavor": "m1.small",                                                                                              |
|                      |   "NodeHostname": "node",                                                                                                  |
|                      |   "NetID": "df4a1975-aa4f-40f5-bd86-f1a10ccbde46",                                                                         |
|                      |   "UpstreamNTP": "' iburst'",                                                                             |
|                      |   "DevMode": "false",                                                                                                      |
|                      |   "PuppetModuleBranch": "master",                                                                                          |
|                      |   "AWS::StackName": "oso-stack",                                                                                           |
|                      |   "AWS::StackId": "arn:openstack:heat::92797818f9724adc8e7b7695028f2a4d:stacks/oso/349e7128-cd15-4333-91ed-dff4b9589a46",  |
|                      |   "KeyName": "admin-key",                                                                                                  |
|                      |   "PuppetModuleURL": "",                                           |
|                      |   "UpstreamDNS": "",                                                                                         |
|                      |   "SubnetID": "ce313fbb-3379-4b47-8949-54c6f54c962a",                                                                      |
|                      |   "NodeFlavor": "m1.small",                                                                                                |
|                      |   "Password": "password",                                                                                                  |
|                      |   "AWS::Region": "ap-southeast-1",                                                                                         |
|                      |   "Prefix": "",                                                                                                 |
|                      |   "BrokerHostname": "broker"                                                                                               |
|                      | }                                                                                                                          |
| stack_name           | oso-stack                                                                                                                  |
| stack_status         | CREATE_COMPLETE                                                                                                            |
| stack_status_reason  | Stack create completed successfully                                                                                        |
| template_description | Template for setting up an OpenShift Origin environment                                                                    |
| timeout_mins         | 60                                                                                                                         |
| updated_time         | 2014-03-07T19:04:06Z                                                                                                       |

Verify the Neutron security-group has been created for the OpenShift stack:

neutron security-group-list
| id                                   | name                                           | description                     |
| 7a342261-a35d-43d6-8b74-c73ae9741e79 | oso-stack-OpenShiftOriginSecurityGroup-2c6rzmfgug65  | OpenShift Origin Firewall Rules |
| a8082e6e-8203-48bc-a211-31b38dd8620b | default                                        | default                         |

Verify the Neutron Broker/Node ports has have created for the OpenShift stack. 'Note: You will see an additional port that has been provisioned by Neutron for the DHCP Agent.

neutron port-list
| id                                   | name                         | mac_address       | fixed_ips                                                                             |
| 45d81f77-fad9-48e7-97d6-1dbe3b7d9102 | oso-stack-NodePort-lmqxyringm6p  | fa:16:3e:c7:ca:93 | {"subnet_id": "ce313fbb-3379-4b47-8949-54c6f54c962a", "ip_address": ""} |
| 8665eab6-d267-463d-9723-46c2d2d8bf54 | oso-stack-BrokerPort-3zy5ng4hcr73| fa:16:3e:b2:bb:36 | {"subnet_id": "ce313fbb-3379-4b47-8949-54c6f54c962a", "ip_address": ""} |
| abb46c5d-08cf-4d17-917c-57b6cba937d7 |                          | fa:16:3e:22:f6:db    |  {"subnet_id": "ce313fbb-3379-4b47-8949-54c6f54c962a", "ip_address": ""} |

Verify the Broker instance has been spawned and shows an ACTIVE status. Note: The Node instance does not get spawned until the Broker has been spawned and is fully configured:

nova list
| ID                                   | Name                             | Status | Task State | Power State | Networks                 |
| 439cca5f-9b61-46c8-90d4-065d315af15e | oso-stack-BrokerInstance-eml774uhi2hg  | ACTIVE | None    | Running  | public223= |

You should then be able to SSH into your Broker instance. Keep in mind that you need to specify the SSH private key that was used to create the Nova key-pair. ec2-user is the default user name. You can change the default user name in heat.conf (instance_user=ec2-user).

ssh -i ~/.ssh/id_rsa ec2-user@<BROKER_IP>
sudo -i

You can view the installation log at /var/log/configure_openshift.log

tail -f /var/log/configure_openshift.log

Note:' You can safely ignore the following warning messages that may appear during your Puppet run:
Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults
Warning: Augeas[network-scripts](provider=augeas): Loading failed for one or more files, see debug for /augeas//error output

The puppet run should complete with the following message:

Notice: Finished catalog run in xxx seconds

You can also view the Heat provisioning log.

less /var/log/heat-provision.log

Note: The last line in the log should be Provision done: 2014-03-07 18:47:09.791809

The broker is now successfully deployed by Heat. You can repeat these verification steps for the Node instance if you would like. Otherwise, wait 10-15 minutes for Heat to complete the rest of the stack deployment and issue the heat stack-list or heat stack-show <STACK_NAME> commands to verify the successful completion of the stack:

heat stack-list
| id                                   | stack_name | stack_status       | creation_time        |
| 349e7128-cd15-4333-91ed-dff4b9589a46 | oso-stack  | CREATE_COMPLETE    | 2014-03-07T18:41:07Z |

Deploy Your First Application

At this point, your OpenShift stack should have been successfully deployed:

heat stack-list
| id                                   | stack_name | stack_status       | creation_time        |
| 349e7128-cd15-4333-91ed-dff4b9589a46 | oso-stack  | CREATE_COMPLETE    | 2014-03-07T18:41:07Z |

If not, do not proceed with this section and follow the Deployment Verification for troubleshooting assistance. Otherwise, follow the Deploy Your First Application Section of the OpenShift Automated Deployment Guide.




This document is based on the following:

  • OpenStack Heat Wiki [1]
  • OpenStack Documentation [[2]
  • OpenShift Origin Comprehensive Deployment Guide [3]
  • OpenShift Origin User’s Guide [4]
  • OpenShift Example PHP Readme [5]


Daneyon Hansen

Rating: 0.0/5 (0 votes cast)

Personal tools