Additional troubleshooting information for Unified Expert Advisor 7.6.1

From DocWiki

(Difference between revisions)
Jump to: navigation, search
 
(One intermediate revision not shown)
Line 1: Line 1:
-
'''Back''': [[Troubleshooting Cisco Unified Expert Advisor]]
+
 
 +
----
 +
'''Back''': [[Troubleshooting Unified Expert Advisor]]
'''View/Add tips for Release 7.6(1): [[:Category:Unified Expert Advisor, Release 7.6(1)]]'''
'''View/Add tips for Release 7.6(1): [[:Category:Unified Expert Advisor, Release 7.6(1)]]'''

Latest revision as of 23:15, 2 July 2009


Back: Troubleshooting Unified Expert Advisor

View/Add tips for Release 7.6(1): Category:Unified Expert Advisor, Release 7.6(1)

Contents

Problem Solving Process

The following steps provide some guidelines to assist in isolating a particular issue.

  1. Analyze the problem and create a clear problem statement. Define symptoms and potential causes.
  2. Gather the facts that you need to help isolate possible causes.
  3. Consider the Guidelines to Assist in Isolating Problems (below) based on the facts that you gathered.
  4. Create an action plan. Begin with the most likely problem and devise a plan in which you manipulate only one variable at a time.
  5. Implement the action plan, performing each step while testing to see whether the symptom disappears.
  6. Analyze the results to determine whether the problem has been resolved. If it has, the process is complete.
  7. If the problem has not been resolved, create an action plan based on the next most probable cause on your list, or contact the Cisco Technical Assistance Center (TAC), or your Cisco Partner.
Note Note: Only change one variable at a time. If that does not resolve the issue, undo that change and move on to the next step of your plan.

Guidelines to Assist in Isolating Problems

  • Was anything recently added, removed, or modified?
  • Is it a reproducible event?
  • Does it occur at a particular time of day, or day of week?
  • Have there been any changes made to the domain, network, or security policies?


Troubleshooting Checklist

Complete this checklist to assist in isolating the issue, or to provide information to your support partner or Cisco Technical Support.

  1. What is the version of Unified Expert Advisor that is currently running? Include any patch or upgrade information.
  2. Is this a new installation or an upgrade?
  3. If this is an upgrade, what version was previously installed?
  4. When did the problem occur?
  5. What are the observed symptoms, and the conditions under which these symptoms occur?
  6. Was anything changed or updated in hardware, software, or network components prior to the first occurrence of the observed symptoms?
  7. Describe the related call flow. Some examples include: Public Switched Telephone Network(PSTN) originated or IP Phone originated.
  8. Is the problem reproducible?
  9. What is the call transfer method used?
  10. Are you able to capture a screen shot of the error or failure? If Yes, save it to a file and attach to a case.


Network Topology

Complete this checklist to assist in isolating the issue, or to provide information to Cisco Technical Support.

  1. Has auto-negotiate been disabled on all PCs, routers, and switch ports?
    Note Note: Duplex/speed mismatch between a device and its corresponding port on the switch is the single most common problem for network latency.
  2. Is a network topology diagram available?
  3. Which type of IP Gateway is being used in this Unified Expert Advisor solution?
  4. On which server are the recorded media files located, and what is the path to those files?
  5. Collect and provide versions of IOS, applications, and Engineering Special (ES)/patch levels in the environment.

Obtaining Log Files

Log files are obtained through the Real Time Monitoring Tool (RTMT) Plugin. You obtain the plugin from the operations console.

Downloading and Installing the RTMT Plugin

The RTMT plugin can be downloaded from the operations console. Versions of the plugin are available for both Windows and Linux platforms. To download and install the RTMT plugin:

  1. Log in to the operations console for Unified Expert Advisor.
  2. Select Tools > RTMT Plugin Downloads.
  3. Select the platform (Windows or Linux) then click Download. The plugin is downloaded to your local computer. The file size of the executable is around 35 MB.
  4. Double-click the plugin executable on your local computer to install it. Details on the installation can be found in Chapter 2 of the Real Time Monitoring Tool Administration Guide for Cisco Unified Expert Advisor.

Obtaining Log Files Using the Real Time Monitoring Tool

Use the RTMT tool to browse, view, and download log files. This task explain show to obtain the logs for the runtime server(s).

  1. Start the RTMT tool. From Windows, select Start > Programs > Cisco > CallManager Serviceability > Real-time Monitoring Tool.
  2. Log in with the username and password you created for Unified Expert Advisor while installing Unified Expert Advisor.
  3. Select the Default Profile.
  4. In the left pane, labeled System, Click Trace & Log Central.
  5. In Trace & Log Central, double-click Remote Browse. A dialog appears.
    Note Note: More details about Remote Browse can be found in the Real Time Monitoring Tool Administration Guide for Cisco Unified Expert Advisor.
  6. In the dialog, select the radio button for Trace Files. Click Next.
  7. Select the checkbox next to Expert Advisor Runtime Service. You can check either the box for All Servers to view the logs for all runtime servers in this cluster, or you can check the box for individual servers. Click Next.
  8. You can optionally select System Services/Applications to obtain logs for additional system services, or just click Finish to obtain just the runtime server logs.
  9. A new pane appears in Trace and Log Central with a folder called Nodes in it. Double click the Nodes folder and drill down to the runtime servers. Double-click the runtime folder todisplay the list of log files in the right pane. There are several log files that appear in the right page, including: MMCA-runtime.[TIMESTAMP].log, MMCA-runtime.[TIMESTAMP].startup.log , MMCA.[TIMESTAMP].out, Error-runtime.[TIMESTAMP].startup.log, and Various zipped version of the above files.
  10. Double-click the latest version of MMCA-runtime.[TIMESTAMP].log to open it in the default viewer. This file contains a majority of the call-logging for the runtime server. You can optionally select the file and click Download at the bottom of the page.
    Note Note: Zip files must be downloaded and unzipped locally with a zip program to be viewed, theycannot be viewed from within the RTMT tool.

Trace Definitions

This section lists the trace definitions for the infrastructure, runtime server, reporting server, ORM server, OAMP server, and common subsystems.

Infrastructure Trace Definitions

This section lists the infrastructure trace definitions for subsystems.

Trace Description
TRACE_HANDLED_EXCEPTION Description of the exception and how it was handled
TRACE_JMX JMX and management interface related traces
TRACE_JMS JMS and message bus related traces
TRACE_HEARTBEAT Related to heartbeats, heartbeat thread, or heartbeat send/received
TRACE_PARAM For any parameters (not just method arguments)
TRACE_CALL For traces related to a call / call processing
TRACE_MESSAGE For general debug details of incoming/outgoing messages
TRACE_NOTIFICATION Trace for notification API
TRACE_GENERAL_CFG General traces for config API
TRACE_OOOQUEUE Set this bit to enable OoOQueue tracing
TRACE_METHOD When entering/exiting a method
TRACE_LOW_LEVEL Bits and Bytes, etc

Runtime Server Subsystem Trace Definitions

This section lists the Infrastructure Trace Definitions for Subsystems.

Agent State Monitoring (ASM)

Trace Description
TRACE_ASM Details on Agent State Monitoring

Reporting Adaptor (RA)

Trace Description
CONFIG Trace configuration
MSG_PUBLISHING Trace message publishing
RAI_MESSAGE Trace MPI messages
CEI_MESSAGE Trace CEI Messages
SYSTEM Trace global activity
REI_MESSAGE Trace REI messages
REPORTING_ADAPTER Trace reporting adapter

Intelligent Call Manager Gateway (ICMGW)

Trace Description
CONNECTION ACMI connection management tracing, including OPEN_REQ/OPEN_CONF, and CLOSE_REQ/CLOSE_CONF
AGENT_STATE_UPDATE Agent State Update Trace Mask
AUTO_CONFIG Auto-config Trace Mask
ROUTING Routing Trace Mask
ILLEGAL Illegal Message Trace Mask
REPORTING Reporting Trace Mask
OAMP OAMP Trace Mask
HEARTBEAT Reporting Trace Mask

Work Assigner (WA)

Trace Description
TRACE_CONTACT Trace everything associated with a contact
TRACE_MATCHING Trace all match processing
TRACE_RESOURCE Trace everything associated with a resource
TRACE_SENDMSGS Trace all messages sent
TRACE_RECVMSGS Trace all messages received
TRACE_DEBUG Trace all debug events
TRACE_STATE Trace all state processing
TRACE_QUEUING Trace all queuing activity
TRACE_TRACKINGOBJECT Trace everything associated with a tracking object
TRACE_CONFIG Trace all configuration related activity
TRACE_TIMER Trace all timer related events
TRACE_ADMIN Trace all admin events and responses

Resource Manager (RM)

Trace Description
ENDPOINT For the Endpoint API tracing
MPI_METHOD_ENTRY/EXIT When entering/exiting a method in the MPI channel
MPI_MESSAGE For general debug details of incoming/outgoing messages for MPI channel
RDI_METHOD_ENTRY/EXIT When entering/exiting a method in the RDI channel
ASSIGNMENT_QUEUE_ENTRY/EXIT For the Assignment Queue API method entry/exit tracing
CMI_MESSAGE For general debug details of incoming/outgoing messages for CMI channel
RESOURCE_METHOD_ENTRY/EXIT For the Resource/Agent API method entry/exit tracing
REI_MESSAGE For general debug details of incoming/outgoing messagesfor REI channel
TASK_METHOD_ENTRY/EXIT For the Task API method entry/exit tracing
CMI_METHOD_ENTRY/EXIT When entering/exiting a method in the CMI channel
TASK Traces for the Tasks in the core API.
CONFIG_METHOD_ENTRY/EXIT When entering/exiting a method in the RDI channel
RESOURCE Traces for Resources in the core API
RDI_MESSAGE For general debug details of incoming/outgoing messagesfor RDI channel
INTERACTION_METHOD_ENTRY/EXIT For the Interaction API method entry/exit tracing
INTERACTION For the Interaction API tracing
MPI_CALL For traces related to a call / call processing inside the MPIchannel
MPI_EVENTS For internal events processed inside the MPI channel
REI_METHOD_ENTRY/EXIT When entering/exiting a method in the REI channel
WRI_EVENTS For internal events processed inside the WRI channel
WRI_MESSAGE For general debug details of incoming/outgoing messagesfor WRI channel
RESOURCE_EVENTS For internal events posted or dispatched by the ResourceAPI
MESSAGE For general debug details of incoming/outgoing messages
REI_EVENTS For internal events processed inside the RDI channel
ASSIGNMENT_QUEUE Traces for Assignment Queues in the core API
ENDPOINT_METHOD_ENTRY/EXIT For the Endpoint API method entry/exit tracing
WRI_METHOD_ENTRY/EXIT When entering/exiting a method in the WRI channel

Contact Manager (CM)

Trace Description
CONFIG Trace config activity
BRE_MESSAGE Trace BRE messages
CONTACT_MANAGER Trace contact activity
MPI_MESSAGE Trace MPI messages
SYSTEM Trace global activity
CONTACT Trace contact activity
KB_MESSAGE Trace KB messages
PARTICIPANT Trace participant activity
WA_MESSAGE Trace Work Assigner messages
ICM_MESSAGE Trace ICM Gateway messages
CONTACT_DETAIL Trace CONTACT_DETAIL activity

Media Platform Interface (MPI)

Only set the traces in this section with the assistance of Cisco.

Trace
SIP_STACK_DSUTIL_THREAD
SIP_STACK_AUTHENTICATION
SIP_STACK_DSSIPLLAPI_RESOLVER
SIP_STACK_DSSIPMLAPI_CALLSTATE
SIP_STACK_DSSIPLLAPI_TRANSACTIONMANAGEMENT_ACK
SIP_STACK_DSSIPLLAPI_TRANSACTIONMANAGEMENT_PRACK
SIP_STACK_DSSIPMLAPI_CALLMANAGEMENT
SIP_STACK_CONFIG
SIP_STACK_DSSIPLLAPI_CONNECTION
SIP_STACK_DSSIPLLAPI_LlSM
SIP_STACK_DSSIPOBJECT_HEADER
SIP_STACK_DSSIPMLAPI_REGISTRATION
SIP_STACK_DSSIPLLAPI_TRANSACTIONMANAGEMENT
SIP_STACK_DSSIPREFER_REFER
SIP_STACK_DUMP
SIP_STACK_DSSIPLLAPI_LlSM_CLIENT
SIP_STACK_DSSIPDIALOG_OFFERANSWER
SIP_STACK_DSSIPMIME_MIME
MPI_CALL_TRACE
MPI_METHOD_TRACE
SIP_STACK_DSSIPLLAPI_LlSM_SERVER_SWITCHSTATE
SIP_STACK_DSUTIL_DSMESSAGESTATISTICS
SIP_STACK_DSSIPHLAPI_HLCALL
SIP_STACK_DSSIPLLAPI_CONNECTIONMANAGEMENT
SIP_STACK_DSSIPLLAPI_LlSM_CLIENT_SWITCHSTATE
SIP_STACK_DSSIPLLAPI_LlSM_CLIENT_TIMERS
SIP_STACK_DSSIPLLAPI_PERF
SIP_STACK_DSSIPEVENTS_EVENTS
SIP_STACK_DSSIPLLAPI_LlSM_CLIENT_USERCB
SIP_STACK_DSSIPLLAPI_WIRE
MPI_HANDLED_EXCEPTION_TRACE
SIP_STACK_DSSIPHLAPI_HLCALLMANAGEMENT
SIP_STACK_DSSIPLLAPI_TRANSACTIONMANAGEMENT_CANCE
MPI_PARAM_TRACE
SIP_STACK_DSSIPLLAPI_TRANSACTIONMANAGEMENT_REQUEST
MPI_LOW_LEVEL_TRACE
SIP_STACK_DSUTIL_SOCKET
SIP_STACK_DSSIPLLAPI_TRANSACTIONKEY
SIP_STACK_DSSIPLLAPI_TRANSACTIONMANAGEMENT_RESPONSE
SIP_STACK_EXCEPTION
SIP_STACK_DSSIPLLAPI_LlSM_SERVER_USERCB
SIP_STACK_DSSIPLLAPI_LlSM_SERVER_TIMERS
SIP_STACK
SIP_STACK_DSSIPOBJECT_MESSAGE
SIP_STACK_DSSIPLLAPI_LlSM_SERVER

Resource Desktop Adaptor (RDA)

Trace Description
CONFIG Config trace bit is used to trace the RDA configuration issue with the database including loading, updating, deleting the message set, initial setup configuration properties, expert advisor configuration properties.
IM_ACTIVITY IM activity is a place holder for future IM activity. Currently, no trace is used by this trace bit.
MPI_MESSAGE MPI message trace bit is used to trace the RDA JMSmessage exchange between the MPI layer.

MPI message trace bit is used to trace the RDA JMSmessage exchange between the MPI layer. MPI layer including the agent presence subscription and notification, system user publication, registration to the presence service.It also indicates the condition of the IM message exchange between the expert advisor and the system.

SYSTEM System trace bit is used to trace the RDA subsystem status such as whether the system is in partial service, out of service or in service It also trace the error condition like topic issue.
REI_MESSAGE RDI message trace bit is used to trace the RDA JMSmessage exchange between the REI protocol.
PRESENCE_ACTIVITY PRESENCE_ACTIVITY trace bit is a place holder for future PRESENCE activity. Currently, no trace is used by this trace bit.
PARSER PARSER trace bit is place holder for any PARSER activity.Currently, no trace is used by this trace bit.
RDI_MESSAGE RDI message trace bit is used to trace the RDA JMSmessage exchange between the RDI protocol.

RDI message trace bit is used to trace the RDA JMSmessage exchange between the RDI protocol. It mainly indicates the condition of the IM message exchange between the expert advisor with the RM subsystem including offerTaskRequest, re-prompting, taskAssignCmd and resource state change request.

Reporting Server Subsystem

Reporting Subsystem (RS)

Trace Description
EXTRA_DEBUG Only set this trace with the assistance of Cisco
DETAILED_DEBUG Only set this trace with the assistance of Cisco
DEBUG Only set this trace with the assistance of Cisco


ORM Server Subsystem Trace Definitions

Trace Description
ORM (OAMP Resource Manager) Only set these traces with the assistance of Cisco.

OAMP Server Subsystem Trace Definitions

Trace Description
TRACE_BULK To control logging for looping/bulk operations
TRACE_GENERAL_UI For tracing the general OAMP UI
TRACE_EXCEPTION For tracing Exceptions
TRACE_PARAM For tracing Parameters
TRACE_DBACCESS Trace DB Access for db fetch and modify such as Save,Update, Delete
TRACE_METHOD For tracing of Entry/Exit of Methods

Common Subsystem Trace Definitions

Infrastructure

Trace Description
TRACE_STATS Operations of the Stats Manager
TRACE_SERVICEABILITY Traces to do serviceability, the act of logging and tracing
TRACE_THREAD All Infrastructure thread operations
TRACE_SNMP TRACE_SNMP SNMP Forwarder/logging/stats
TRACE_SHUTDOWN Log detailed shutdown info
TRACE_LICENSING Log any/all licensing operations
TRACE_STARTUP Log detailed startup info
LOAD_SUBSYSTEM When loading subsystems
TRACE_TIMER Logs when a Timer expires

OAMP_BO

Trace Description
TRACE_BULK To control logging for looping/bulk operations
TRACE_EXCEPTION For tracing Exceptions
TRACE_GENERAL_BO General Traces for OAMP back-end
TRACE_PARAM For tracing Parameters
TRACE_DBACCESS Trace DB Access for db fetch and modify such as Save,Update, Delete
TRACE_METHOD For tracing of Entry/Exit of Methods

Rating: 0.0/5 (0 votes cast)

Personal tools