Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting with ACE Logging
From DocWiki
m |
m |
||
Line 4: | Line 4: | ||
|align="center"|'''Guide Contents''' | |align="center"|'''Guide Contents''' | ||
|- | |- | ||
- | |[[Cisco Application Control Engine (ACE) Troubleshooting Guide|Main Article]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Overview of ACE Troubleshooting|Overview of ACE Troubleshooting]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Understanding the ACE Module Architecture and Traffic Flow|Understanding the ACE Module Architecture and Traffic Flow]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Preliminary ACE Troubleshooting|Preliminary ACE Troubleshooting]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting ACE Boot Issues|Troubleshooting ACE Boot Issues]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting with ACE Logging|Troubleshooting with ACE Logging]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Connectivity|Troubleshooting Connectivity]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Ethernet Ports|Troubleshooting ACE Appliance Ethernet Ports]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Remote Access|Troubleshooting Remote Access]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Access Control Lists|Troubleshooting Access Control Lists]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Network Address Translation|Troubleshooting Network Address Translation]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting ACE Health Monitoring|Troubleshooting ACE Health Monitoring]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Layer 4 Load Balancing|Troubleshooting Layer 4 Load Balancing]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Layer 7 Load Balancing|Troubleshooting Layer 7 Load Balancing]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Redundancy|Troubleshooting Redundancy]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting SSL|Troubleshooting SSL]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Compression|Troubleshooting Compression]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Performance Issues|Troubleshooting Performance Issues]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- ACE Resource Limits|ACE Resource Limits]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Managing Resources|Managing ACE Resources]]<br> | + | |[[Cisco Application Control Engine (ACE) Troubleshooting Guide|Main Article]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Overview of ACE Troubleshooting|Overview of ACE Troubleshooting]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Understanding the ACE Module Architecture and Traffic Flow|Understanding the ACE Module Architecture and Traffic Flow]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Preliminary ACE Troubleshooting|Preliminary ACE Troubleshooting]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting ACE Boot Issues|Troubleshooting ACE Boot Issues]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting with ACE Logging|Troubleshooting with ACE Logging]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Connectivity|Troubleshooting Connectivity]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Ethernet Ports|Troubleshooting ACE Appliance Ethernet Ports]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Remote Access|Troubleshooting Remote Access]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Access Control Lists|Troubleshooting Access Control Lists]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Network Address Translation|Troubleshooting Network Address Translation]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting ACE Health Monitoring|Troubleshooting ACE Health Monitoring]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Layer 4 Load Balancing|Troubleshooting Layer 4 Load Balancing]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Layer 7 Load Balancing|Troubleshooting Layer 7 Load Balancing]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Redundancy|Troubleshooting Redundancy]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting SSL|Troubleshooting SSL]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Compression|Troubleshooting Compression]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Performance Issues|Troubleshooting Performance Issues]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- ACE Resource Limits|ACE Resource Limits]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Managing Resources|Managing ACE Resources]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Show Counter Reference|Show Counter Reference]]<br> |
|} | |} | ||
Latest revision as of 21:28, 11 March 2011
This article describes the ACE system logging facility, how to enable logging, and how to use system messages as troubleshooting tools.
Contents |
Overview of ACE System Logging
The ACE provides a system logging (syslog) facility that collects and saves system messages and outputs them to destinations that you specify as follows:
- Buffer
- Console
- Flash memory
- Host (remote syslog server)
- Monitor
- Standby
- Supervisor
Each virtual context generates logs independently from the other virtual contexts. The admin virtual context does not log on behalf of the other contexts in the ACE.
The ACE logs the following connection setup and teardown messages in the CP at the connection speed: 106023, 302022, 302023, 302024, and 302025. These setup and teardown syslogs are directly forwarded to a remote server. Because of the potentially large number of these messages or if you require high-rate system logging of connection setup and teardown messages, use the logging fastpath command. This command disables CP syslogs and enables logging of these messages through the DP using a slightly different format and the following syslog IDs: 106028, 302028, 302029, 302030, and 302031, respectively. You can log these messages through the fast path only to an external syslog server. All other enabled logging destinations are disabled by the logging fastpath command.
You can limit the rate at which the ACE generates syslog messages by using the logging rate-limit command. This command allows you to rate limit syslogs based on one of the following criteria:
- Time interval
- Logging level
- Message ID
Besides logging system messages, the ACE logs access control list (ACL) deny entries.
For details about ACE system message logging, see the Cisco Application Control Engine Module System Message Guide (Software Version A2(1.0)).
Enabling ACE Logging
To enable logging on the ACE module and send syslogs to the monitor, enter the following commands:
ACE_module5/Admin(config)# logging enable ACE_module5/Admin(config)# logging monitor 7 ACE_module5/Admin(config)# logging trap 7 ACE_module5/Admin(config)# no logging message 111008 ACE_module5/Admin(config)# no logging message 111009 ACE_module5/Admin(config)# logging timestamp ACE_module5/Admin(config)# do terminal monitor
Use the logging monitor severity_level command only when you are troubleshooting problems on the ACE or when there is minimal load on the network. Using this command at other times when the ACE is active may degrade performance.
![]() | Note: | logging trap defines the severity sent to the syslog server. |
![]() | Note: | If you do not see syslog messages on the console after enabling logging with the logging enable and logging monitor 7 commands, log out of the ACE and then log in again. |
To enable logging to a syslog server, use the following command syntax:
logging host ip_address [tcp | udp [/port#]] | [default-udp] | [format emblem]
![]() | Note: | If you specify the default-udp option and TCP logging fails, the ACE sends logging messages over UDP. |
You can verify that the ACE defaults to UDP by entering the following command:
ACE_module5/Admin# show logging Syslog logging: enabled Facility: 20 History logging: disabled Trap logging: enabled (level - debugging) Timestamp logging: disabled Fastpath logging: disabled Persist logging: disabled Standby logging: disabled Rate-limit logging: disabled (min - 0 max 100000 msgs/sec) Console logging: disabled Monitor logging: disabled Logging to 5.1.0.40 tcp/514 default-udp (sending on UDP) Device ID: disabled Message logging: none Buffered logging: enabled (level - debugging) maximum size 681984 Buffer info: current size - 681984 global pool - 1048576 used pool - 1048576 min - 0 max - 681984 cur ptr = 42894 wrapped - yes
Use the logging supervisor command to allow the aggregation of critical syslogs from multiple virtual devices to the Catalyst 6500 series switch or to the Cisco 7600 series router syslog. For example, enter the following command:
ACE_module5/Context(config)# logging supervisor ? <0-7> 0-emerg;1-alert;2-crit;3-err;4-warn; 5-notif;6-inform;7-debug cat6k# show logging . . . cat6k#17w3d: %TRINITY-7-TRINITY_SYSLOG_DEBUG: %ACE-7-111009: User 'admin' executed cmd: show running.
Logging Severity Levels
The severity_level argument specifies the maximum level for system log messages sent to the console. The severity level that you specify indicates that you want syslog messages at that level and messages less than the level. For example, if the specified level is 3, the syslog displays level 3, 2, 1, and 0 messages. We recommend that you use a lower severity level, such as 3, since logging at a high rate may impact the performance of the ACE.
Allowable entries are as follows:
- 0—emergencies (System unusable messages)
- 1—alerts (Take immediate action)
- 2—critical (Critical condition)
- 3—errors (Error message)
- 4—warnings (Warning message)
- 5—notifications (Normal but significant condition)
- 6—informational (Information message)
- 7—debugging (Debug messages)
Adding Information to syslogs
After you have enabled system message logging and have specified a destination for the system messages, you can add more information to the system messages that may be helpful in troubleshooting issues with your ACE module. For example, you can do the following:
- Add a timestamp
- Identify the messages sent to a syslog server
- Identify the ACE device ID in messages that are sent to a syslog server
To add a timestamp to syslog messages, enter the following command:
ACE_module5/Admin(config)# logging timestamp
To identify messages that are sent to a syslog server by severity level, enter the following command:
ACE_module5/Admin(config)# logging trap severity_level
For example, to identify the ACE device ID in messages that are sent to a syslog server, use the following command syntax:
ACE_module5/Admin(config)# logging device-id {context-name | hostname | ipaddress interface_name | string text}
Troubleshooting ACE Logging
The commands in the following sections are useful for troubleshooting the system message logging facility.
Displaying Logging Statistics
ACE_module5/Admin# show logging statistics Syslog statistics: sent 349 discarded 64 Messages sent: console 0 buffer 348 persistent 0 supervisor 1 history 0 monitor 0 host 0 misc 0 Messages discarded: cfg rate-limit 0 hard rate-limit 0 server down 5 queue full 59 errors 0 SNMP-related counters: notifications sent 0 history table flushed 0 messages ignored 0 NP-related counters: to-CP dropped 0 fastpath sent 0 fastpath dropped 0
ACE_module5/Admin# show logging queue Logging Queue length limit : 80 msg(s), 59 msg(s) discarded. Current 0 msg on queue, 80 msgs most on queue CP messages received: 426 , 59 msg(s) discarded. IXP messages received: 82 Xscale messages received: 0 System Max Queue size: 20080 System Free Queue size for allocation: 19920
In the above example, the ACE has discarded 59 control plane (CP) messages. By default, the syslog message queue can hold 80 messages. You can increase the size of the syslog message queue by using the logging queue command in configuration mode. Set the queue size before you start collecting syslog messages. When traffic is heavy, messages may be discarded if the queue size is too small. The maximum number of messages that the queue can hold is 8192.
Displaying the Logging History
To display the ACE logging history, enter the following command from the console:
ACE_module5/Admin# show logging history syslog_trinity_show_history for context 0: 1(Mar 24 2009 16:39:36): from "KERN" ACE-5-111008:User 'admin' executed the 'logging history 7' command. 2(Mar 24 2009 16:39:48): from "KERN" ACE-5-111008:User 'admin' executed the 'logging console 7' command. 3(Mar 24 2009 16:39:56): from "KERN" ACE-7-111009:User 'admin' executed cmd: do sho logging history 4(Mar 24 2009 16:49:50): from "KERN" ACE-7-111009:User 'admin' executed cmd: do show logging message 5(Mar 24 2009 16:51:35): from "KERN" ACE-4-405001:Received ARP REQUEST collision from 10.1.1.240 00.0c.29.74.51.fa on interface vlan100 6(Mar 24 2009 16:51:36): from "KERN" ACE-4-405001:Received ARP RESPONSE collision from 10.1.1.240 00.0b.fc.fe.1b.03 on interface vlan100 7(Mar 24 2009 16:51:40): from "KERN" ACE-4-405001:Received ARP REQUEST collision from 10.1.1.240 00.0c.29.74.51.fa on interface vlan100 8(Mar 24 2009 16:51:41): from "KERN" ACE-4-405001:Received ARP RESPONSE collision from 10.1.1.240 00.0b.fc.fe.1b.03 on interface vlan100 9(Mar 24 2009 16:54:46): from "KERN" ACE-7-111009:User 'admin' executed cmd: telnet 10.1.1.130
Displaying Logging Messages
If a particular system message does not appear in the syslog history, check that the message is enabled and that the logging level is correct for that message. To display the default logging level and the current status of a system message, all system messages, or disabled system messages, enter the following command:
ACE_module5/Admin# show logging message message_id | all | disabled
For example, to display all disabled system messages in the ACE, enter the following command:
ACE_module5/Admin# show logging message disabled Message logging: message 111008: default-level 5 (disabled) message 111009: default-level 7 (disabled)
Displaying Logging Persistence
The logging persistence command enables logging to disk0: in Flash memory. The messages are stored in a subdirectory of disk0: called /messages.
To display logging persistence, enter the following command:
ACE_module5/Admin# show logging persistent Persist info: current size - 6626 global pool - 1048576 used pool - 6626 min - 0 max - 189582 cur ptr = 6638 wrapped - no Mar 24 2009 09:51:31 Admin: %ACE-4-405001: Received ARP REQUEST collision from 10.1.1.240 00.0c.29.74.51.fa on interface vlan100 Mar 24 2009 09:51:32 Admin: %ACE-4-405001: Received ARP RESPONSE collision from 10.1.1.240 00.0b.fc.fe.1b.03 on interface vlan100 Mar 24 2009 09:51:36 Admin: %ACE-4-405001: Received ARP REQUEST collision from 10.1.1.240 00.0c.29.74.51.fa on interface vlan100 Mar 24 2009 09:51:37 Admin: %ACE-4-405001: Received ARP RESPONSE collision from 10.1.1.240 00.0b.fc.fe.1b.03 on interface vlan100
Displaying the Logging Rate Limit
To display the logging rate limit, enter the following command:
ACE_module5/Admin# show logging rate-limit Rate-limit logging: (min - 0 max 100000 msgs/sec) 100000 messages 1 seconds level 7