System Encountered A Delay In Getting Data

From DocWiki

Jump to: navigation, search


Contents

Scenario Setup

File Server

  1. Edit egpl_master.properties file to change "controllerreq.timeout" to 6 seconds

Application Server

  1. Stop/Start Cisco Service to take effect

Problem Statement

Unable to start the Archive process. Pop-up forces administrators to "Log Out Now."

Log Out is Not Required!

Log in as SA and try to start the Archive process from the System Console. You should receive the error before the process starts successfully.

Although the only option in the pop-up may appear to be Log Out Now, a user can click the X to close the pop-up and wait for the application to catch up. This is commonly seen when starting processes in the System Console. If the Java process does not finish starting before the default timeout of 60 seconds, the pop-up will be displayed. Notice the bottom progress bar still shows "Starting process" and the status says "Stopped"

ProcessStartup.png


Once the hourglass changes to a normal mouse cursor, the process says "Running," and the bottom progress bar shows "Ready," we can simply close the pop-up and continue on with our session.

Ready.png


Why the delay? Will the logs explain it?

This issue is occurring for all processes, so it is more of an application-wide issue. Individual process logs will not be useful.

Logging in as SA requires only the core services (DSM Controller, DSM Registry Server) to be able to access the console - and of course, an Application Server. Start with those logs and look for "timeout" to find your start process request.

2011-06-23 04:35:01.506 GMT-0400 <@> ERROR <@> [405:ajp-eimApp.eim.lab%2F14.104.48.16-15006-3] <@> ProcessId:3076 <@> PID:50 <@> UID:12 <@> HttpSessionId:  <@> logtimeouterror <@>  <@> Timeout error occurred for the controller:http://eimweb/system/web/view/platform/system/../../../controller/platform/system/process/startprocess.jsp, view from where the controller request was sent:list <@> 

But where is this timeout set? One of the core property files to the system: C:\CIM\eService\config\egpl_master.properties The timeout message in the log states the field being used: "controller request" --> "controllerreq"

egpl_master.properties

We can see in the properties file that controllerreq.timeout is set extremely low, and would explain why we saw timeouts when starting processes.

Controllerreq.png


The default timeout of 60 seconds can be increased, but keep in mind that this is typically only masking a much larger problem. It should be considered little more than a band-aid until a more in-depth investigation can occur.

  1. Open C:\CIM\eService\config\egpl_master.properties
  2. If administrators are receiving the messages during normal admin activities, increase "controllerreq.timeout" from 60 to something higher (i.e. 120).
  3. Note that the "action.timeout" variable was removed in 4.3(2). Timeout of agent activities is managed by the application itself after 4.3(1).
  4. Save and reboot all Application Servers in the deployment for the change to take effect.

Resolution

The majority of the time this issue occurs, it is due to network latency between the Application/Web servers and agents, or between the EIM/WIM components themselves. (such as a slow response from the database, or a heavily-used process with DEBUG-level logging).

A network audit would be a great recommendation for repeated but sporadic occurrences after property files have been verified.

Rating: 5.0/5 (1 vote cast)

Personal tools