Logger Checklist

From DocWiki

Revision as of 10:54, 21 September 2010 by Saychakr (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Observations / Issues Initial Analysis CheckList Resolution Steps/Data Collection Inputs
Logger Recovery Issue + Purge Issues + Historical data missing/deleted 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it. 5) Collect and analyze the Recovery logs from both the sides of Logger. Tracing: EMSTraceMask should be set to 'FFFF'. 6) If there is a problem with the copying of historical logger from one side of logger to other, we've to analyze the Hlgr logs also with EMSTraceMask set to 'FFFF'. 7) If there is a missing in the records/records are getting deleted, check for the purge settings from the registry/run through the set up to check the purge settings for each table. Note: If we want to check for some particular historical/half hour data, we've have to collect the OPC and Router logs also. Tracing should be corresponding to the data missing (Eg: TCD missing, we've to enable closedcall tracing in OPC) 8) If the customer is seeing that they are the data is exceeding the DB limit (Eg: 80%), they have to increase the Database size and the Log size. 9) Data Holes after Recovery in Post Upgrade: CSCtd87838

Replication Issue+Data not replicating from Logger to HDS+Replication process crashing 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it. 6) Collect the Replication logs from both the sides of Logger and the distributor. Tracing: EMSTraceMask should be set to 'FFFF' Note: If we want to check for some particular historical/half hour data, we've have to collect the OPC and Router logs also. Tracing should be corresponding to the data missing (Eg: TCD missing, we've to enable closedcall tracing in OPC)+Hlgr logs with EMSTraceMask set to 'FFFF'

Historical logger Issue 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it with the stack trace. 5) Collect HLGR logs with EMSTraceMask set to 'FFFF' Note: If we want to check for some particular historical/half hour data, we've have to collect the OPC and Router logs also. Tracing should be corresponding to the data missing (Eg: TCD missing, we've to enable closedcall tracing in OPC) 6) Hlgr crashes - FileTimeToSystemTime fail - CSCsm84234 & lb-hlgr Fail: ICRDb::ParseECCBinDataAndInsertTerminationCallVariable, FileTimeToDosDateTime() failed - CSCtf45693.

Configuration Logger Issue. Configuration data 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it. 5) Collect Clgr logs with EMSTraceMask set to 'FFFF' 6) If the issue is corresponding to some configuration update, check the status of Router and collect the Router logs with 'Config Changes' and 'Config Delivery' traces enabled. 7) To synchronize the configuration data between the side A and side B loggers, do a manual synchronization using ICMDBA. 8) If the foreign data is getting garbled in the database, check the registry HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSSQLServer\Client\DB-Lib\AutoAnsiToOem is turned on or not, it has to be turned on. Reference 613499361.

CICM Replication Issue 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it. 5) Collect Crpl logs with EMSTraceMask set to 'FFFF' 6) Configuration In NAM Side.

  • Application Gateway
  • Instance Explorer
  • Routing client should match in the NIC explorer

In CICM Side.

  • INCRP NIC.
  • Routing client should match in the NIC explorer

7) Check the NAM-CICM setup/config document for their configuration. 8) Collect the uaw logs from the cicm distributor with EMSTraceMask set to 'FFFF'

Data Holes in Logger or HDS Please go through the below cases:

1) http://wwwin-tools.cisco.com/casekwery/getServiceRequest.do?clientTimeZoneOffset=-5.5&id=613769521+&header=Y&status=I&view=D

2) http://topic.cisco.com/news/cisco/cs/cse-dev-cc/dsc05118.html#05268

3) http://wwwin-tools.cisco.com/casekwery/getServiceRequest.do?clientTimeZoneOffset=-5.5&id=610110517+&header=Y&status=I&view=D

Loggers not in sync Historical data: We've to collect the hlgr and rcv logs with EMSTraceMask set to 'FFFF'. Troubleshoot from the logs.

Configuration data: We can do a synchronization from the icmdba tool.

Reporting Issues (data not correct in database). Real time as well as Historical data 1) Check if it's the historical report or the real time report which is getting effected.

2) If the historical report has the problem, ensure that the HDS is pointing to the primary distributor. 3) In both the type of reporting, check the origin of the data (whether OPC cuts or is it generated by Router). Then make sure that the data is passed properly. 4) If confirmed that the data is coming to Logger DB/HDS DB (historical data) & AWDB (realtime data) properly from the logs. but not inserting into the destination DB, collect the below set of logs and Table export proving the problem. 5) If the historical data is missing, analyze the OPC logs (enable Closedcalls if it's TCD) with correct tracing, Router logs, HLGR (EMSTraceMask set to 'FFFF'), RCV (EMSTraceMask set to 'FFFF'), Logger RPL (EMSTraceMask set to 'FFFF') & Distributor RPL (EMSTraceMask set to 'FFFF'). 6) If real time data is missing, analyze OPC and Router logs with RealTime tracing enabled plus, RTS, RTD, RTC and UAW logs all with EMSTraceMask set to 'FFFF'. 7) If customer has doubts/clarification regarding particular fields in tables, please check the schema help for the defenition and also check Webview help to understand the field corresponding to the Table in the report.

AW/HDS Real Time Client Issue 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it. 5) Collect RTC, RTS and RTD logs with EMSTraceMask set to 'FFFF' 6) Check if the customer is running into CSCso12705 (Stack trace: ExpInterlockedPopEntrySListFault+0) or CSCsu47823 (Stack trace: 00415D35 015FFF60 ESQLScriptRealTime::delete_all_records+145 00416150 015FFF84 rt_script_image_handling+300 7C34940F 015FFFB8 endthread+AA 77E6608B 015FFFEC GetModuleFileNameA+EB ) where we used to get plenty of cases. 7) If some real time data is missing or getting changed, we've to collect the Router logs with RealTime tracing enabled and also the OPC logs with RealTime traces enabled.

Real Time Server Issue 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it. 5) Collect RTC, RTS and RTD logs with EMSTraceMask set to 'FFFF' 6) If some real time data is missing or getting changed, we've to collect the Router logs with RealTime tracing enabled and also the OPC logs with RealTime traces enabled. 7) Stack Trace: 77E4BEE7 0CC3FE24 RaiseException+3C 7C359AED 0CC3FE64 CxxThrowException+34 7C1C6939 0CC3FE78 Ordinal1185+19 00405632 0CC3FEAC RealTimeServer::QueueBaseRecordType+352 004069B5 0CC3FEEC RealTimeServer::QueueBaseRecords+275 004070B6 0CC3FF5C RealTimeServer::RegisterDistributor+406 004071F6 0CC3FF78 RealTimeServer::ReadClientThread+56 00407516 0CC3FF84 ReadClientThread+16 7C34940F 0CC3FFB8 endthread+AA 77E64829 0CC3FFEC GetModuleHandleA+DF - 610925807

Real Time Distributor Issue 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it. 5) Collect RTC, RTS and RTD logs with EMSTraceMask set to 'FFFF' 6) If some real time data is missing or getting changed, we've to collect the Router logs with RealTime tracing enabled and also the OPC logs with RealTime traces enabled.

Distributor Replication Issue 1) Check for the ICM Version

2) Any ES/ET applied 3) Is there a crash, if yes, pass on the exe and pdb files from icm\bin folder along with mdmp file. 4) Check in the topic search if there are previous TAC cases and defects associated to it. 5) Collect the Replication logs from both distributor and Logger with EMSTraceMase set to 'FFFF' 6) If the customer wants to move HDS from one drive the another, check the SR 608403437 for steps.

Others 1) Problem: Updateaw process keeps crashing when attempting to connect to Logger DB. Solution:the Last update Key (LUK) must match in CC router memory, awcontrol table in awdb database, and the max(RecoveryKey) in Config_Message_Log table in logger db.

2) Problem: UAW assertion failure in module CCDatabase::GetHostIPAdresses. Solution: Changing references of Logger and Router servers to IP addresses instead of hostname.

Rating: 5.0/5 (1 vote cast)

Personal tools