OpenStack: Troubleshooting

From DocWiki

Jump to: navigation, search

This page contains a sortable table of common problems and solutions, as well as tips on how to troubleshoot your OpenStack deployment both during and after installation.

For a complete list of bugs, see:


Common Issues and Solutions

The following table contains commonly asked questions, known issues, and bugs. Puppet Warning and Error messages are included. Solutions and workarounds are provided where possible.

To sort by Release, Scenario, or Component, click the sort icon ( Sort none icon.gif ) under the heading name. "Component" includes all OpenStack components, plus Puppet and Cobbler.

Cisco OpenStack Problems and Solutions
Release Scenario Component Issue Solution
i.0 and earlier All in One Swift Puppet error: Swift swift-container-replicator, swift-container-sync, or swift-account-replicator will not start. For example, Puppet logs the following message: Error: Could not start Service[swift-container-replicator]: Execution of '/sbin/start swift-container-replicator' returned 1 Start Swift manually: service swift-container-replicator restart;service swift-container-sync restart;service swift-account-replicator restart or do a second Puppet catalog run. See Bug #1274358 and Bug #1326390
Any Full HA Ceph Ceph-backed cloud is not working properly. Perform a second Puppet catalog run.
Any Any Neutron Neutron server does not start when using the Cisco plugin. Start neutron-server manually. See data/hiera_data/user.common.yaml for instructions. This occurs because Ubuntu init scripts assume only one Neutron plugin config file is used.
Any Any Neutron Neutron's Firewall as a Service does not start. When starting the Neutron level 3 agent, include the following argument: --config-file /etc/neutron/fwaas_driver.ini .
h.2 and later Any Nova Live migration does not work with NFS storage. Ensure that the Nova user ID and GID is the same on all systems.
Any Any Cobbler My OpenStack nodes do not have a default gateway. To enable default routes, remove the netcfg/no_default_route=true \ line from /etc/puppet/data/cobbler/cobbler.yaml on your build node and reapply puppet to update Cobbler. The default Cobbler configuration assumes that OpenStack nodes are isolated behind the build server.
Any 2 Role Puppet The Puppet run is very short and contains the message: Warning: The role is not defined for scenario: 2_role. Check for the build node line in the /etc/puppet/data/role_mappings.yaml file: your_build_node_hostname: build. Modify or add the line if necessary. Verify that your_build_node_hostname exactly matches the name of your incipient build node: facter hostname Finally, run Puppet apply again: puppet apply -v /etc/puppet/manifests/site.pp
h.1 and earlier All in One Puppet Warning message: You cannot collect exported resources without storeconfigs being set This message should disappear if you do another Puppet catalog run. See Bug #1282281.
Any Any Swift Warning message: swift storage server $service must specify $service-server This message is harmless; ignore it. See Bug #1289187.
h.3 and earlier Any Keystone Warning message: keystone_host, keystone_port and keystone_scheme are deprecated. Use keystone_url instead This message is harmless; ignore it.
h.3 and earlier Any Nagios Warning about nagios services restarting. This message is harmless; ignore it. The message is thrown by the example configuration of nagios that is installed by default.
h.3 and earlier Full HA MongoDB Warning about mongodb in HA scenarios. This message is harmless; ignore it. In HA scenarios, Mongodb will throw warnings until all three nodes are up, a quorum is formed, and a primary node elected, at which point the warnings will stop.
Any Any Cobbler Warning message: Unrecognised escape sequence '\+' in file /usr/share/puppet/modules/coi/manifests/profiles/cobbler_server.pp. This message is harmless; ignore it. See Bug #1238118.
i.0 Multi-node Neutron VMs in a GRE network do not receive IP addresses via DHCP. Fixed in i.1.

See Bug #1327294

h.2 and earlier All in One Graphite Error: graphite-manage syncdb --noinput returned 1 instead of one of [0]

Error: /Stage[main]/Graphite/Exec[graphite-syncdb]/returns: change from notrun to 0 failed: graphite-manage syncdb --noinput returned 1 instead of one of [0]

Perform a second Puppet catalog run. See Bug #1282239
i.0 High Availability Keystone Error: /Stage[main]/Keystone::Roles::Admin/Keystone_role[_member_]: Could not evaluate: Execution of '/usr/bin/keystone --os-endpoint http://x.x.x.x:35357/v2.0/ role-list' returned 1: An unexpected error prevented the server from fulfilling your request. (HTTP 500) Synch the Keystone database manually:

keystone-manage db_sync. See Bug #1319432

i.0 Any Nova ProgrammingError: (ProgrammingError) (1146, "Table '' doesn't exist") Create the database manually:

nova-manage db sync then restart services. See Bug #1326069

i.0 High Availability Cobbler Error messages of the form: Error: cobbler-ubuntu-import precise-x86_64 returned 1 instead of one of [0] Copy the file data/hiera_data/hostname/build_server.yaml to build_server_name.yaml in the same directory, where build_server_name is the short name of your build server.

Reading Puppet Logfiles

Puppet generates a large volume of messages during a catalog run, but most of the time only error and warning messages are of concern. Errors will stop the Puppet run; warnings will not. Many of the warnings seen during puppet runs are harmless. See Common Issues and Solutions for common Puppet warning messages and their meanings.

Using OpenStack Logs for Troubleshooting

The logfiles for OpenStack services are one of the primary tools for troubleshooting problems with OpenStack.

Most OpenStack logfiles are named following this convention:


For example, logs from the glance-api service are found in:


Logs from the nova-api service are found in:


If you don't see a log for the service you're trying to troubleshoot, look in:


A log file is sometimes created there instead.

It may help in troubleshooting to enable verbose (more detailed diagnostic messages) logging or debug (lower level traces) logging for an OpenStack service or for all OpenStack services.

To enable verbose or debug logging for an OpenStack service:

  1. Edit the service's configuration file.
  2. In the file, set the verbose or debug boolean to True.
  3. Save the file.
  4. Restart the service.

Note: Some OpenStack services might not have verbose or debug flags. Instead, you can try enabling verbose or debug logging for all OpenStack services, below.

To enable verbose or debug logging for all OpenStack services:

  1. Edit data/hiera_data/common.yaml
  2. In the file, set the verbose or debug flag to true: debug: true and verbose: true
  3. Save the file.
  4. Do a Puppet catalog run or restart Puppet.

Rating: 5.0/5 (1 vote cast)

Personal tools