Cisco Application Control Engine (ACE) Troubleshooting Guide -- Show Counter Reference -- Command Set 5
From DocWiki
(Difference between revisions)
m |
|||
| Line 1: | Line 1: | ||
| + | {| align="right" border="1" cellspacing = "0" | ||
| + | |align="center"|'''Guide Contents''' | ||
| + | |- | ||
| + | |[[Cisco Application Control Engine (ACE) Troubleshooting Guide|Main Article]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Overview of ACE Troubleshooting|Overview of ACE Troubleshooting]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Understanding the ACE Module Architecture and Traffic Flow|Understanding the ACE Module Architecture and Traffic Flow]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Preliminary ACE Troubleshooting|Preliminary ACE Troubleshooting]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting ACE Boot Issues|Troubleshooting ACE Boot Issues]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting with ACE Logging|Troubleshooting with ACE Logging]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Connectivity|Troubleshooting Connectivity]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Ethernet Ports|Troubleshooting ACE Appliance Ethernet Ports]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Remote Access|Troubleshooting Remote Access]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Access Control Lists|Troubleshooting Access Control Lists]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Network Address Translation|Troubleshooting Network Address Translation]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting ACE Health Monitoring|Troubleshooting ACE Health Monitoring]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Layer 4 Load Balancing|Troubleshooting Layer 4 Load Balancing]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Layer 7 Load Balancing|Troubleshooting Layer 7 Load Balancing]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Redundancy|Troubleshooting Redundancy]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting SSL|Troubleshooting SSL]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Compression|Troubleshooting Compression]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Troubleshooting Performance Issues|Troubleshooting Performance Issues]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- ACE Resource Limits|ACE Resource Limits]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Managing Resources|Managing ACE Resources]]<br>[[Cisco Application Control Engine (ACE) Troubleshooting Guide -- Show Counter Reference|Show Counter Reference]]<br> | ||
| + | |} | ||
| + | |||
| + | __TOC__ | ||
| + | |||
== show probe detail == | == show probe detail == | ||
Latest revision as of 21:50, 11 March 2011
show probe detail
Displays configuration information and statistics for a probe.
Sample Output
ACE30001/rlb_ssg# show probe detail
probe : generic-https-probe
type : HTTPS
state : ACTIVE
description :
----------------------------------------------
port : 443 address : 0.0.0.0 addr type : -
interval : 30 pass intvl : 30 pass count : 3
fail count: 3 recv timeout: 10
http method : GET
http url : /index.html
conn termination : GRACEFUL
expect offset : 0 , open timeout : 10
expect regex : -
send data : -
--------------------- probe results --------------------
probe association probed-address probes failed passed health
------------------- ---------------+----------+----------+----------+-------
serverfarm : https-test
real : FC8-server[0]
192.168.2.165 19634 0 19634 SUCCESS
Socket state : CLOSED
No. Passed states : 1 No. Failed states : 0
No. Probes skipped : 0 Last status code : 0
No. Out of Sockets : 0 No. Internal error: 0
Last disconnect err : -
Last probe time : Fri Sep 25 09:40:39 2009
Last fail time : Never
Last active time : Fri Sep 18 14:03:15 2009
Notes
For a description of the output fields, see chapter "Configuring Health Monitoring" in the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.
show proc
Displays general information about all processes running on the ACE. This command and "show processes" are identical.
Sample Output
ACE30001/Admin# show proc
PID State PC Start_cnt TTY Process
----- ----- -------- ----------- ---- -------------
1 S 2ac1964c 1 - init
2 S 0 1 - keventd
3 S 0 1 - ksoftirqd_CPU0
4 S 0 1 - ksoftirqd_CPU1
5 S 0 1 - kswapd
6 S 0 1 - bdflush
7 S 0 1 - kupdated
72 S 2ac2968c 1 - cron
116 S 0 1 - loop0
117 S 0 1 - kjournald
123 S 0 1 - loop1
124 S 0 1 - kjournald
130 S 0 1 - loop2
131 S 0 1 - kjournald
137 S 0 1 - loop3
138 S 0 1 - kjournald
144 S 0 1 - loop4
145 S 0 1 - kjournald
306 S 0 1 - mts_kutil
385 S 0 1 - Pkt_Fifo_Tx
386 S 0 1 - Pkt_Fifo_Rx
387 S 0 1 - Pkt_Fifo_Util
400 S 0 1 - Netio_Encap
556 S 2ac5b64c 1 - telnetd
557 S 2b650f64 1 1 vsh
612 S 2c9f6fb0 1 - lcpfw
625 S 0 1 - cavium
733 S 2abe068c 1 - n2_perf_stats
742 S 0 1 - ipcp_pci_rx
759 S 0 1 - keventd
784 S 801010b4 1 - insmod
839 S 2b68364c 1 - httpd
854 S 2b2d364c 1 - sysmgr
872 S 2ceb56b4 1 - syslogd
873 S 2aebd64c 1 - sdwrapd
877 S 2affffb0 1 - pfmgr
881 S 2cd8e64c 1 - ntp
882 S 2acf064c 1 - lmgrd
883 S 2b0a564c 1 - fs-daemon
884 S 2ceb3020 1 - syslogd
885 S 2ceb56b4 1 - syslogd
886 S 2afb264c 1 - confcheck
888 S 2b14d984 1 - licmgr
895 S 2acf064c 1 - cisco
896 S 2b174984 1 - vshd
898 S 2ceb3020 1 - syslogd
899 S 2aec664c 1 - ntpd
901 S 2b68f8e4 1 - httpd
902 S 2b680fb0 1 - httpd
906 S 2d4f364c 1 - xinetd
907 S 2cb1afb0 1 - vacd
908 S 2ae7664c 1 - ttyd
909 S 2ae4e87c 1 - sysinfo
910 S 2dd956b4 1 - snmpd
911 S 2aec0fb0 1 - sme
912 S 2b1a364c 1 - scripted_hm
913 S 2cf6064c 1 - radiusd
914 S 2ae2afb0 1 - pktcap
915 S 2adfafb0 1 - nat_dnld
916 S 2cdb9020 1 - itasca_ssl
917 S 2afabfb0 1 - itasca_route_mgr
918 S 2af5cfb0 1 - itasca_fm
919 S 2aff1fb0 1 - ifmgr
920 S 2b019fb0 1 - hsrp_track
921 S 2b43a6b4 1 - hm
922 S 2ca2f64c 1 - ha_mgr
923 S 2af8164c 1 - ha_dp_mgr
924 S 2b01064c 1 - gslb_proto
925 S 2b0ae64c 1 - dhcrelay
926 S 2af5a64c 1 - core-dmon
927 S 2c9be6b4 1 - config_cntlr
928 S 2e2bee58 1 - cfgmgr
929 S 2ae69fb0 1 - bpdu
930 S 2aed4984 1 - ascii-cfg
931 S 2cecb020 1 - arp_mgr
932 S 2adf0020 1 - aclmerged
933 S 2ce9364c 1 - tacacs
934 S 2cf1264c 1 - ldap
935 S 2cf0f984 1 - aaa
939 S 2b438020 1 - hm
941 S 2c9bc020 1 - config_cntlr
942 S 2c9be6b4 1 - config_cntlr
943 S 2cecb020 1 - arp_mgr
944 S 2cecb020 1 - arp_mgr
945 S 2b43a6b4 1 - hm
946 S 2b43a6b4 1 - hm
947 S 2cecb020 1 - arp_mgr
948 S 2b43a6b4 1 - hm
949 S 2e3a0020 1 - cfgmgr
951 S 2b43a6b4 1 - hm
952 S 2ce9464c 1 - securityd
953 S 2ab15024 1 - cfgmgr
954 S 2cdb9020 1 - itasca_ssl
955 S 2cdbb6b4 1 - itasca_ssl
956 S 2cdba984 1 - itasca_ssl
957 S 2cdbb6b4 1 - itasca_ssl
960 S 2e2bee58 1 - cfgmgr
961 S 2e3696dc 1 - cfgmgr
962 S 2adf0020 1 - aclmerged
963 S 2e3a0020 1 - cfgmgr
964 S 2adb96dc 1 - aclmerged
965 S 2e3a0020 1 - cfgmgr
966 S 2dd93020 1 - snmpd
967 S 2dd94984 1 - snmpd
968 S 2dd956b4 1 - snmpd
1132 S 0 1 - Peer
1146 S 2ac0f87c 1 - klogd
1794 S 2ac5b64c 1 - telnetd
1795 S 2b68a64c 1 0 vsh
1858 S 0 1 - Peer
4866 S 2b689984 1 1 vsh
4867 S 2ac9f87c 1 1 more
4868 R 2ac6087c 1 - ps
20999 S 2b68a64c 1 S00 vsh
- NR - 0 - installer
- NR - 0 - session_agent
Notes
The process state value can be one of:
- D – Uninterruptible sleep (usually I/O related)
- ER – Error while running
- NR – Not running
- R – Running or runnable (on run queue)
- S – Interruptible sleep (waiting for an event to complete)
- T – Stopped, either by a job control signal or because it is being traced
- W – Paging
- X – Process is dead
- Z – Defunct ("zombie") process; terminated but not reaped by its parent
show processes cpu
Displays CPU information for the Intel Pentium processor. This command is used for troubleshooting high CPU issues, slow response times on the CLI and performance issues.
Sample Output
ACE30001/Admin# show proc cpu
CPU utilization for five seconds: 2%; one minute: 5%; five minutes: 5%
PID Runtime(ms) Invoked uSecs 1Sec 5 Sec 1 Min 5 Min Process
----- ----------- -------- ----- ----- ---- ---- ---- -----------
1 13737 120542 113 0.0 0.0 % 0.0 % 0.0 % init
2 0 32 17 0.0 0.0 % 0.0 % 0.0 % keventd
3 172 10815 15 0.0 0.0 % 0.0 % 0.0 % ksoftirqd_CPU0
4 0 12 9 0.0 0.0 % 0.0 % 0.0 % ksoftirqd_CPU1
5 0 1 10 0.0 0.0 % 0.0 % 0.0 % kswapd
6 0 1 11 0.0 0.0 % 0.0 % 0.0 % bdflush
7 567639 294614 1926 0.0 0.0 % 0.2 % 0.10% kupdated
72 1154 10005 115 0.0 0.0 % 0.0 % 0.0 % cron
116 109 185 591 0.0 0.0 % 0.0 % 0.0 % loop0
117 20 188 108 0.0 0.0 % 0.0 % 0.0 % kjournald
123 90 247 367 0.0 0.0 % 0.0 % 0.0 % loop1
124 755 285 2650 0.0 0.0 % 0.0 % 0.0 % kjournald
130 148 1900 78 0.0 0.0 % 0.0 % 0.0 % loop2
131 83 1990 41 0.0 0.0 % 0.0 % 0.0 % kjournald
137 27 46 590 0.0 0.0 % 0.0 % 0.0 % loop3
138 1 33 44 0.0 0.0 % 0.0 % 0.0 % kjournald
144 391 9856 39 0.0 0.0 % 0.0 % 0.0 % loop4
145 435 9912 43 0.0 0.0 % 0.0 % 0.0 % kjournald
306 27264 718916 37 0.0 0.0 % 0.0 % 0.0 % mts_kutil
385 9627 558762 17 0.0 0.0 % 0.0 % 0.0 % Pkt_Fifo_Tx
386 278879 8841533 31 0.0 0.6 % 0.2 % 0.1 % Pkt_Fifo_Rx
387 0 1 5 0.0 0.0 % 0.0 % 0.0 % Pkt_Fifo_Util
400 1 20 68 0.0 0.0 % 0.0 % 0.0 % Netio_Encap
556 516 7287 70 0.0 0.5 % 0.1 % 0.0 % telnetd
557 1511 2040 741 0.0 0.9 % 0.1 % 0.0 % vsh
612 372224 1315018 283 0.0 0.1 % 0.2 % 0.1 % lcpfw
625 461272 58972997 7 0.0 0.4 % 0.3 % 0.0 % cavium
733 242945 584082 415 0.0 0.1 % 0.1 % 0.1 % n2_perf_stats
742 160 1901 84 0.0 0.0 % 0.0 % 0.0 % ipcp_pci_rx
759 0 1 7 0.0 0.0 % 0.0 % 0.0 % keventd
784 33075 117935 280 0.0 0.0 % 0.0 % 0.0 % insmod
839 21434 589851 36 0.0 0.0 % 0.0 % 0.0 % httpd
854 108727 1192089 91 0.0 0.10% 0.2 % 0.0 % sysmgr
872 3863585 58964891 65 0.0 0.31% 0.32% 0.29% syslogd
873 97 39 2504 0.0 0.0 % 0.0 % 0.0 % sdwrapd
877 1368 9864 138 0.0 0.0 % 0.0 % 0.0 % pfmgr
881 20740 127451 162 0.0 0.0 % 0.0 % 0.0 % ntp
882 108 13 8353 0.0 0.0 % 0.0 % 0.0 % lmgrd
883 95 47 2029 0.0 0.0 % 0.0 % 0.0 % fs-daemon
884 4394 293365 14 0.0 0.0 % 0.0 % 0.0 % syslogd
885 59625 604007 98 0.0 0.0 % 0.0 % 0.0 % syslogd
886 173 24 7222 0.0 0.0 % 0.0 % 0.0 % confcheck
888 8246 117982 69 0.0 0.0 % 0.0 % 0.0 % licmgr
895 3726 9845 378 0.0 0.0 % 0.0 % 0.0 % cisco
896 10652 74169 143 0.0 0.0 % 0.0 % 0.0 % vshd
898 64927 9827441 6 0.0 0.0 % 0.0 % 0.0 % syslogd
899 60989 589699 103 0.0 0.0 % 0.0 % 0.0 % ntpd
901 43 48 896 0.0 0.0 % 0.0 % 0.0 % httpd
902 23 47 508 0.0 0.0 % 0.0 % 0.0 % httpd
906 11593 118067 98 0.0 0.0 % 0.0 % 0.0 % xinetd
907 50695 61125 829 0.0 0.0 % 0.0 % 0.0 % vacd
908 12475 294862 42 0.0 0.0 % 0.0 % 0.0 % ttyd
909 4088 60169 67 0.0 0.90% 0.13% 0.2 % sysinfo
910 113389 1188841 95 0.0 0.0 % 0.0 % 0.0 % snmpd
911 19414 293510 66 0.0 0.0 % 0.0 % 0.0 % sme
912 124 59 2103 0.0 0.0 % 0.0 % 0.0 % scripted_hm
913 18147 589714 30 0.0 0.0 % 0.0 % 0.0 % radiusd
914 63 34 1858 0.0 0.0 % 0.0 % 0.0 % pktcap
915 160 291 553 0.0 0.0 % 0.0 % 0.0 % nat_dnld
916 732 10171 72 0.0 0.0 % 0.0 % 0.0 % itasca_ssl
917 6675 118193 56 0.0 0.0 % 0.0 % 0.0 % itasca_route_mgr
918 280 9878 28 0.0 0.0 % 0.0 % 0.0 % itasca_fm
919 43134235 355623 121292 38.5 0.0 % 3.57% 3.68% ifmgr
920 265 9840 26 0.0 0.0 % 0.0 % 0.0 % hsrp_track
921 3966 39373 100 0.0 0.0 % 0.0 % 0.0 % hm
922 55367 254374 217 0.0 0.0 % 0.0 % 0.0 % ha_mgr
923 5570 59293 93 0.0 0.0 % 0.0 % 0.0 % ha_dp_mgr
924 77 44 1752 0.0 0.0 % 0.0 % 0.0 % gslb_proto
925 10551 118031 89 0.0 0.0 % 0.0 % 0.0 % dhcrelay
926 95 49 1949 0.0 0.0 % 0.0 % 0.0 % core-dmon
927 5270 59172 89 0.0 0.0 % 0.0 % 0.0 % config_cntlr
928 711 96 7406 0.0 0.0 % 0.0 % 0.0 % cfgmgr
929 4460 541616 8 0.0 0.0 % 0.0 % 0.0 % bpdu
930 6203 73746 84 0.0 0.0 % 0.0 % 0.0 % ascii-cfg
931 8678273 9613436 902 1.3 0.82% 0.73% 0.72% arp_mgr
932 2340 10199 229 0.0 0.0 % 0.0 % 0.0 % aclmerged
933 29562 590176 50 0.0 0.4 % 0.0 % 0.0 % tacacs
934 21326 589708 36 0.0 0.0 % 0.0 % 0.0 % ldap
935 33096 591425 55 0.0 0.2 % 0.0 % 0.0 % aaa
939 5541 293370 18 0.0 0.0 % 0.0 % 0.0 % hm
941 5812 293361 19 0.0 0.0 % 0.0 % 0.0 % config_cntlr
942 79 44 1805 0.0 0.0 % 0.0 % 0.0 % config_cntlr
943 4725 293357 16 0.0 0.0 % 0.0 % 0.0 % arp_mgr
944 55097 1111282 49 0.0 0.0 % 0.0 % 0.0 % arp_mgr
945 13398 40204 333 0.0 0.0 % 0.0 % 0.0 % hm
946 13377 39867 335 0.0 0.0 % 0.0 % 0.0 % hm
947 70507 338432 208 0.0 0.0 % 0.0 % 0.0 % arp_mgr
948 0 3 59 0.0 0.0 % 0.0 % 0.0 % hm
949 5412 293367 18 0.0 0.0 % 0.0 % 0.0 % cfgmgr
951 0 3 41 0.0 0.0 % 0.0 % 0.0 % hm
952 21731 589850 36 0.0 0.0 % 0.0 % 0.0 % securityd
953 0 1 395 0.0 0.0 % 0.0 % 0.0 % cfgmgr
954 5859 293372 19 0.0 0.0 % 0.0 % 0.0 % itasca_ssl
955 0 2 107 0.0 0.0 % 0.0 % 0.0 % itasca_ssl
956 0 1 385 0.0 0.0 % 0.0 % 0.0 % itasca_ssl
957 0 1 174 0.0 0.0 % 0.0 % 0.0 % itasca_ssl
960 2312 1521 1520 0.0 0.0 % 0.0 % 0.0 % cfgmgr
961 18032 583809 30 0.0 0.0 % 0.0 % 0.0 % cfgmgr
962 5054 293360 17 0.0 0.0 % 0.0 % 0.0 % aclmerged
963 154 147 1052 0.0 0.0 % 0.0 % 0.0 % cfgmgr
964 9595 293365 32 0.0 0.0 % 0.0 % 0.0 % aclmerged
965 53701 16022 3351 0.0 0.0 % 0.0 % 0.0 % cfgmgr
966 4743 293360 16 0.0 0.0 % 0.0 % 0.0 % snmpd
967 4023 7706 522 0.0 0.0 % 0.0 % 0.0 % snmpd
968 199527 5897126 33 0.0 0.1 % 0.1 % 0.1 % snmpd
1132 0 4 119 0.0 0.0 % 0.0 % 0.0 % Peer
1146 13 26 514 0.0 0.0 % 0.0 % 0.0 % klogd
1794 38 127 299 0.0 0.0 % 0.0 % 0.0 % telnetd
1795 355 103 3454 0.0 0.0 % 0.0 % 0.0 % vsh
1858 16111 118034 136 0.0 0.0 % 0.0 % 0.0 % Peer
4898 9 5 1873 0.0 0.0 % 0.0 % 0.0 % vsh
4899 20 5 4127 0.0 0.0 % 0.0 % 0.0 % more
4900 122 6 20451 0.0 0.0 % 0.0 % 0.0 % ps
20999 2150 65452 32 0.0 0.0 % 0.0 % 0.0 % vsh
Notes
This command output format is:
CPU utilization for five seconds: 12%; one minute: 4%; five minutes: 4% PID Runtime(ms) Invoked uSecs 1Sec 5 Sec 1 Min 5 Min Process ----- ----------- -------- ----- ----- ---- ---- ---- ----------- 864 14560628 141874354 102 0.0 0.25% 0.25% 0.24% syslog
The noteworthy statistics are the CPU averages for the last 1 second, 5 seconds, 1 minute and 5 minutes, as shown in the first line of the display are the CPU averages for all processes running on the ACE:
"CPU utilization for five seconds: 12%; one minute: 4%; five minutes: 4%"
Thereafter, each task is listed individually with its CPU averages.
show processes memory
Displays current memory allocation per process.
Sample Output
ACE30001/Admin# show proc memory
PID MemAlloc StackBase/Ptr Process
----- -------- ----------------- ----------------
1 14592 7fff7f40/7fff77d0 init
2 0 0/0 keventd
3 0 0/0 ksoftirqd_CPU0
4 0 0/0 ksoftirqd_CPU1
5 0 0/0 kswapd
6 0 0/0 bdflush
7 0 0/0 kupdated
72 20928 7fff7e10/7fff7b90 cron
116 0 0/0 loop0
117 0 0/0 kjournald
123 0 0/0 loop1
124 0 0/0 kjournald
130 0 0/0 loop2
131 0 0/0 kjournald
137 0 0/0 loop3
138 0 0/0 kjournald
144 0 0/0 loop4
145 0 0/0 kjournald
306 0 0/0 mts_kutil
385 0 0/0 Pkt_Fifo_Tx
386 0 0/0 Pkt_Fifo_Rx
387 0 0/0 Pkt_Fifo_Util
400 0 0/0 Netio_Encap
556 17824 7fff7ce0/7fff7870 telnetd
557 190628 7fff7e30/7fff6ab8 vsh
612 184064 7fff7e70/7fff7a00 lcpfw
625 0 0/0 cavium
733 8144 7fff7e40/7fff6ac8 n2_perf_stats
742 0 0/0 ipcp_pci_rx
759 0 0/0 keventd
784 105880 7fff7e40/801010c4 insmod
839 838336 7fff7df0/7fff7c70 httpd
854 892560 7fff7e70/7fff7c98 sysmgr
872 250656 7fff7cb0/7fff7aa8 syslogd
873 13960 7fff7ce0/7fff6dd0 sdwrapd
877 112460 7fff7ce0/7fff7990 pfmgr
881 220544 7fff7d00/7fff7840 ntp
882 96472 7fff7c30/7fff6980 lmgrd
883 96656 7fff7d00/7fff7508 fs-daemon
884 250656 7fff7cb0/135d0658 syslogd
885 250656 7fff7cb0/7f7ff4e8 syslogd
886 30768 7fff7d00/7fff7b98 confcheck
888 236140 7fff7ce0/7fff7770 licmgr
895 125720 7fff7c10/7fff7ab8 cisco
896 2896768 7fff7d10/7fff7778 vshd
898 250656 7fff7cb0/7f5ff850 syslogd
899 43888 7fff7cd0/7fff7b90 ntpd
901 973504 7fff7df0/7fff7b60 httpd
902 989888 7fff7df0/7fff7b90 httpd
906 113648 7fff7cb0/7fff7b10 xinetd
907 310484 7fff7ce0/7fff7990 vacd
908 112252 7fff7d10/7fff7660 ttyd
909 30772 7fff7d00/7fff7910 sysinfo
910 1649136 7fff7ce0/7fff6520 snmpd
911 70024 7fff7ce0/7fff7960 sme
912 56676 7fff7cc0/7fff77b0 scripted_hm
913 343360 7fff7cd0/7fff68b0 radiusd
914 21856 7fff7ce0/7fff7990 pktcap
915 457212 7fff7cd0/7fff76a8 nat_dnld
916 224108 7fff7cc0/7fff7978 itasca_ssl
917 2379776 7fff7ca0/7fff7720 itasca_route_mgr
918 55728 7fff7cd0/7fff7988 itasca_fm
919 252336 7fff7ce0/7fff7948 ifmgr
920 23968 7fff7cc0/7fff7938 hsrp_track
921 99112 7fff7cf0/7fff7070 hm
922 78720 7fff7ce0/7fff7740 ha_mgr
923 66848 7fff7cd0/7fff7738 ha_dp_mgr
924 71488 7fff7cc0/7fff7750 gslb_proto
925 96384 7fff7cf0/7fff7968 dhcrelay
926 93168 7fff7cf0/7fff7730 core-dmon
927 409888 7fff7cc0/7fff7718 config_cntlr
928 8482640 7fff7cd0/7fff64a8 cfgmgr
929 56416 7fff7ce0/7fff77a0 bpdu
930 51856 7fff7cf0/7fff78d8 ascii-cfg
931 672272 7fff7cd0/7fff7310 arp_mgr
932 347152 7fff7cd0/7fff7760 aclmerged
933 328636 7fff7cd0/7fff66a0 tacacs
934 267072 7fff7ce0/7fff6af8 ldap
935 182544 7fff7ce0/7fff7b18 aaa
939 99112 7fff7cf0/10039e90 hm
941 409888 7fff7cc0/1005a678 config_cntlr
942 409888 7fff7cc0/7f7ff7e8 config_cntlr
943 672272 7fff7cd0/127557c0 arp_mgr
944 672272 7fff7cd0/7f7ff328 arp_mgr
945 99112 7fff7cf0/7f7ff400 hm
946 99112 7fff7cf0/7f5ff400 hm
947 672272 7fff7cd0/7f5ff828 arp_mgr
948 99112 7fff7cf0/7f3ff400 hm
949 8482640 7fff7cd0/105b6f98 cfgmgr
951 99112 7fff7cf0/7f1ff400 hm
952 153864 7fff7cd0/7fff6b60 securityd
953 8482640 7fff7cd0/7f7ffa68 cfgmgr
954 224108 7fff7cc0/1001baf8 itasca_ssl
955 224108 7fff7cc0/7f7ff340 itasca_ssl
956 224108 7fff7cc0/7f5ff180 itasca_ssl
957 224108 7fff7cc0/7f3ff338 itasca_ssl
960 8482640 7fff7cd0/7f5f5550 cfgmgr
961 8482640 7fff7cd0/7f3fa690 cfgmgr
962 347152 7fff7cd0/10029da0 aclmerged
963 8482640 7fff7cd0/7f1fa870 cfgmgr
964 347152 7fff7cd0/7f7ff8d8 aclmerged
965 8482640 7fff7cd0/7eff0600 cfgmgr
966 1649136 7fff7ce0/10846c4c snmpd
967 1649136 7fff7ce0/7f7ffa70 snmpd
968 1649136 7fff7ce0/7f5ff7d8 snmpd
1132 0 0/0 Peer
1146 59632 7fff7ca0/7fff7be8 klogd
1794 17824 7fff7ce0/7fff7870 telnetd
1795 170148 7fff7e30/7fff6a60 vsh
1858 0 0/0 Peer
4912 190628 7fff7e30/7fff56d8 vsh
4913 5400 7fff7e10/7fff7b20 more
4914 0 7fff7c80/7fff7ad8 ps
20999 198820 7fff7e70/7fff6aa0 vsh
Notes
A statistic that is relevant to troubleshooting in this output is the MemAlloc column. MemAlloc shows the amount of fixed memory allocated by each process. By watching this value, you can discover processes that are using an increasing amount of memory, possibly indicative of a memory usage problem. This would be done by issuing the command at fixed intervals checking for large increases in the allocated memory.
show resource allocation
This output shows resource allocation.
Sample Output
ace19/Admin# sho resource allocation --------------------------------------------------------------------------- Parameter Min Max Class --------------------------------------------------------------------------- acl-memory 0.00% 100.00% default syslog buffer 0.00% 100.00% default conc-connections 0.00% 100.00% default mgmt-connections 0.00% 100.00% default proxy-connections 0.00% 100.00% default bandwidth 0.00% 100.00% default connection rate 0.00% 100.00% default inspect-conn rate 0.00% 100.00% default syslog rate 0.00% 100.00% default regexp 0.00% 100.00% default sticky 0.00% 100.00% default xlates 0.00% 100.00% default ssl-connections rate 0.00% 100.00% default mgmt-traffic rate 0.00% 100.00% default mac-miss rate 0.00% 100.00% default acc-connections 0.00% 100.00% default http-comp rate 0.00% 100.00% default throughput 0.00% 100.00% default
Notes
For more information, see the Managing Resources section of the Troubleshooting Guide.
show resource usage all
When executed from the Admin context, this command displays the resource usage for all other contexts.
Sample Output
ACE30001/Admin# show resource usage all
Allocation
Resource Current Peak Min Max Denied
-------------------------------------------------------------------------------
Context: Admin
conc-connections 9 339 80000 2000000 0
mgmt-connections 18 28 1000 25000 0
proxy-connections 0 118 10486 262144 0
xlates 0 0 10486 262144 0
bandwidth 2072 6920309 5000000 250000000 0
throughput 1058 6907078 5000000 125000000 0
mgmt-traffic rate 1014 13231 0 125000000 0
connections rate 0 2896 10000 250000 0
ssl-connections rate 0 0 10 250 0
mac-miss rate 0 1 20 500 0
inspect-conn rate 0 0 60 1500 0
acl-memory 33920 33920 784000 19652608 0
sticky 1 2 41942 0 0
regexp 0 0 10486 262144 0
syslog buffer 272384 272384 40960 1007616 0
syslog rate 0 14 1000 25000 0
Context: rlb_csg
conc-connections 24 114 2000000 2000000 0
mgmt-connections 0 0 25000 25000 0
proxy-connections 0 11 262144 262144 0
xlates 0 0 262144 262144 0
bandwidth 164 3630 125000000 250000000 0
throughput 164 3630 125000000 125000000 0
mgmt-traffic rate 0 0 0 125000000 0
connections rate 0 32 250000 250000 0
ssl-connections rate 0 0 250 250 0
mac-miss rate 0 0 500 500 0
inspect-conn rate 0 0 1500 1500 0
acl-memory 5520 5520 19650480 0 0
sticky 0 0 1048576 0 0
regexp 14 14 262144 0 0
syslog buffer 0 0 1048576 0 0
syslog rate 0 0 25000 0 0
Context: rlb_ssg
conc-connections 0 11 2000000 2000000 0
mgmt-connections 0 0 25000 25000 0
proxy-connections 0 11 262144 262144 0
xlates 0 0 262144 262144 0
bandwidth 842 12402 125000000 250000000 0
throughput 842 12402 125000000 125000000 0
mgmt-traffic rate 0 0 0 125000000 0
connections rate 10 42 250000 250000 0
ssl-connections rate 0 2 250 250 0
mac-miss rate 10 11 500 500 0
inspect-conn rate 0 0 1500 1500 0
acl-memory 11096 11096 19650480 0 0
sticky 0 0 838 0 0
regexp 18 18 262144 0 0
syslog buffer 1046528 1046528 1048576 0 0
syslog rate 0 10 25000 0 0
Context: spirent_ssg
conc-connections 0 4 2000000 2000000 0
mgmt-connections 0 0 25000 25000 0
proxy-connections 0 4 262144 262144 0
xlates 0 0 262144 262144 0
bandwidth 82 2378 125000000 250000000 0
throughput 82 2378 125000000 125000000 0
mgmt-traffic rate 0 0 0 125000000 0
connections rate 0 25 250000 250000 0
ssl-connections rate 0 0 250 250 0
mac-miss rate 0 0 500 500 0
inspect-conn rate 0 0 1500 1500 0
acl-memory 9456 9456 19650480 0 0
sticky 0 0 838 0 0
regexp 14 14 262144 0 0
syslog buffer 0 0 1048576 0 0
syslog rate 0 0 25000 0 0
Notes
Note the mac-miss rate. Mac-miss messages are sent when the dataplane does not find an encap-entry on a route/mac-lookup and ratelimited is 2k/sec. It's possible to see denied messages if incoming traffic rate is higher than that, since CP can take some time to resolve and program the encap entries. The "mac-miss rate" value then is the number of times ACE got a packet from an unknown mac-address, which the CP needs to learn from these packets to populate the neighbors
show rserver detail
The rserver stats fields are similar to those from show serverfarm detail; the only difference is that "total conn-failures" value appears at the bottom of the output.
Note that this command displays rserver information followed by a separate set of information on every serverfarm (that is, every sfarm-real instance).
Sample Output
ACE30001/Admin# show rserver detail
rserver : 165, type: HOST
state : OPERATIONAL (verified by arp response)
description : -
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : 350000 , out-of-rotation count : 0
bandwidth-rate-limit : 300000000 , out-of-rotation count : 0
weight : 8
---------------------------------
----------connections-----------
real weight state current total
---+---------------------+------+------------+----------+--------------------
serverfarm: IPSEC_GWs
192.168.2.165:0 8 OPERATIONAL 0 2437322
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : - , out-of-rotation count : -
bandwidth-rate-limit : - , out-of-rotation count : -
total conn-failures : 16062
rserver : 231, type: HOST
state : OPERATIONAL (verified by arp response)
description : -
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : 350000 , out-of-rotation count : 0
bandwidth-rate-limit : 300000000 , out-of-rotation count : 0
weight : 8
---------------------------------
----------connections-----------
real weight state current total
---+---------------------+------+------------+----------+--------------------
serverfarm: IPSEC_GWs
192.168.2.231:0 8 OPERATIONAL 0 1188792
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : - , out-of-rotation count : -
bandwidth-rate-limit : - , out-of-rotation count : -
total conn-failures : 6156
rserver : 233, type: HOST
state : OPERATIONAL (verified by arp response)
description : -
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : 350000 , out-of-rotation count : 0
bandwidth-rate-limit : 300000000 , out-of-rotation count : 0
weight : 8
---------------------------------
----------connections-----------
real weight state current total
---+---------------------+------+------------+----------+--------------------
serverfarm: IPSEC_GWs
192.168.2.233:0 8 OPERATIONAL 0 233321
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : - , out-of-rotation count : -
bandwidth-rate-limit : - , out-of-rotation count : -
total conn-failures : 1217
show scp stats
Displays information related to the Switch Module Control Protocol (SCP) statistics. SCP is the protocol that the SUP (IOS) and ACE communicate on over the EOBC channel.
Sample Output
ACE30001/Admin# show scp stats SCP statistics: Tx packets 773704 Rx packets 773704 Tx bytes 195106600 Rx packets 48449378 TX Stats TX Errors 0 TX Timeout 0 Not Connected 0 Tx No memory 0 RX Stats Rx Error 1 Rx No memory 0 Rx no buffers 0 Rx offline 0 Rx message size 0 Rx Kthread enqueue 606026 Rx Kthread dequeue 606026 Rx unknown SAP 0 Rx MTS alloc fail 0 Rx MTS enqueue 606026 Rx MTS enqueue fail 0
Notes
The error-related counters are for the most part self explanatory (see below). In general these errors incrementing would be cause for concern and could be a hardware issue.
There are two main types of errors shown, transmit errors and receive errors. Of note are the following.
- Transmit (TX) Errors:
- timeout — Message sent to SUP timed out and the message was discarded.
- not connected — The EOBC channel between the ACE/SUP is dead. This is a serious error that would most likely result in the ACE not coming up or rebooting without a core.
- no memory — There is no fixed memory available and the message was discarded.
- Receive (RX) Errors:
- no memory — There is no fixed memory available and the message was discarded.
- no buffers — There was no buffer available to receive packets from the SUP and the incoming message is discarded.
- offline — The EOBC channel between the ACE/SUP is dead. This is a serious error that would most likely result in the ACE not coming up or rebooting without a core.
- message size — The incoming message from the SUP exceeded the maximum for the EOBC channel. This indicates a serious communication issue between the SUP and ACE.
show serverfarm detail
Displays statistics for an entire serverfarm, followed by stats for the individual rservers in the serverfarm.
Sample Output
ACE30001/Admin# show serverfarm detail
serverfarm : IPSEC_GWs, type: HOST
total rservers : 3
active rservers: 3
description : -
state : ACTIVE
predictor : LEASTCONNS
slowstart : 0 secs
failaction : -
back-inservice : 0
partial-threshold : 0
num times failover : 0
num times back inservice : 1
total conn-dropcount : 0
---------------------------------
----------connections-----------
real weight state current total failures
---+---------------------+------+------------+----------+----------+---------
rserver: 165
192.168.2.165:0 8 OPERATIONAL 0 2437322 16062
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : - , out-of-rotation count : -
bandwidth-rate-limit : - , out-of-rotation count : -
retcode out-of-rotation count : -
load value : 0
rserver: 231
192.168.2.231:0 8 OPERATIONAL 0 1188792 6156
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : - , out-of-rotation count : -
bandwidth-rate-limit : - , out-of-rotation count : -
retcode out-of-rotation count : -
load value : 0
rserver: 233
192.168.2.233:0 8 OPERATIONAL 0 233321 1217
max-conns : - , out-of-rotation count : -
min-conns : -
conn-rate-limit : - , out-of-rotation count : -
bandwidth-rate-limit : - , out-of-rotation count : -
retcode out-of-rotation count : -
load value : 0
Notes
The serverfarm-related stats are:
| Field | Description |
| serverfarm | Name of the serverfarm and the configured server farm type, HOST or REDIRECT. |
| total rservers | Total number of real servers associated with the server farm. |
| active rservers | Number of real servers that are active in the server farm. |
| description | User-entered text description of the server farm with a maximum of 240 alphanumeric characters. |
| state | Current state of the server farm. Possible values are ACTIVE or INACTIVE. |
| predictor | Configured load-balancing method and values for various fields associated with the predictor. Possible predictor values are:
|
| slowstart | Configured slowstart value. |
| failaction | Action that the ACE takes for connections if a real server fails in a server farm. Possible actions are purge or none. |
| back-inservice | Configured value of the back-inservice keyword of the partial-threshold command. Specifies the minimum percentage of real servers in the primary server farm that must be active again for the ACE to place the server farm back in service. |
| partial-threshold | Configured value of the partial-threshold command. Specifies the minimum percentage of real servers in the primary server farm that must remain active for the server farm to stay up. |
| num times failover | Number of times that the server farm failed over to the backup server farm. |
| num times back inservice | Number of times that the ACE placed the server farm back in service after a failover. |
| total conn-dropcount | Total number of connections that the ACE discarded because the number of connections exceeded the configured conn-limit max value. |
The rserver-related stats are:
| Field | Description |
| Rserver | Name of the real server associated with the server farm. |
| IP Address:Port | IP address and port of the real server. |
| Weight | Weight assigned to the real server in the server farm. |
| State | Current state of the real server. Possible states are OPERATIONAL or OUTOFSERVICE. |
| Current Connections | Number of active connections to the real server. |
| Total Connections | Total number of connections to the specified server farm. |
| Maxconns | Configured maximum allowable number of active connections to a real server. |
| Minconns | Configured minimum number of connections that the number of connections must fall below before sending more connections to a server after it has exceeded the maximum connections threshold. |
| Out-of-rotation-count | Number of times that the real server was not considered for load balancing because the number of connections, connection rate, or bandwidth rate exceeded the configured limits of the server. |
| Conn-rate-limit | Configured connection rate limit of the real server in connections per second. |
| Bandwidth-rate-limit | Configured bandwidth rate limit of the real server in bytes per second. |
| Connections failures | This is incremented if the connection is closed for the following reasons:
An internal exception event can happen for a variety of reasons, such as failure to start or restart a timer, destination NAT failure, redundant connection, etc. |
show service-policy detail
There are two forms of the command:
- show service-policy detail
- show service-policy [name] details
The second displays detailed service-policy counters. The show service-policy [name] command gives a similar though less verbose output for the named service policies.
Sample Output
ACE30001/Admin# show service-policy detail
Policy-map : LB
Status : ACTIVE
Description: -----------------------------------------
Interface: vlan 101
service-policy: LB
class: IPSEC-CLASS
nat:
nat dynamic 1 vlan 102
curr conns : 0 , hit count : 3882870
dropped conns : 388
client pkt count : 23025431 , client byte count: 1548137256
server pkt count : 23003930 , server byte count: 9373406823
conn-rate-limit : - , drop-count : -
bandwidth-rate-limit : - , drop-count : -
VIP Address: Protocol: Port:
192.168.219.150 udp eq 500
192.168.219.150 udp eq 4500
192.168.219.150 50
192.168.219.150 tcp eq 80
loadbalance:
L7 loadbalance policy: IPSEC-GW
VIP Route Metric : 77
VIP Route Advertise : DISABLED
VIP ICMP Reply : ENABLED
VIP State: INSERVICE
curr conns : 0 , hit count : 3882870
dropped conns : 388
client pkt count : 23025431 , client byte count: 1548137256
server pkt count : 23003930 , server byte count: 9373406823
conn-rate-limit : - , drop-count : -
bandwidth-rate-limit : - , drop-count : -
L7 Loadbalance policy : IPSEC-GW
class/match : class-default
LB action: :
sticky group: SeGW_GRP
primary serverfarm: IPSEC_GWs
state: UP
backup serverfarm : -
hit count : 3882870
dropped conns : 0
Parameter-map(s):
UDP-TIMEOUT
Notes
| Field | Description |
| Status | Current operational state of the service policy. Possible states are ACTIVE or INACTIVE. |
| Description | User-entered description of the policy map. |
| Interface | VLAN ID of the interface to which the policy map has been applied |
| Service Policy | Unique identifier of the policy map. |
| Class | Name of the class map associated with the service policy. There can be many classes associated with one service-policy. |
| NAT Dynamic Pool | Identifier of the NAT dynamic pool and the associated VLAN. |
| NAT curr conns | Number of active connections in this NAT Pool. |
| NAT hit count | Number of times a connection was established through this NAT Pool. |
| NAT dropped conns | Number of connections that the ACE discarded because the NAT Pool was out of resources. |
| NAT client pkt count | Number of packets received from the network through this NAT Pool. |
| NAT client byte count | Number of bytes received from the client side network through this NAT Pool. |
| NAT server pkt count | Number of packets received from the service side network through this NAT Pool. |
| NAT server byte count | Number of bytes received from the service side network through this NAT Pool. |
| NAT conn-rate-limit drop-count | Number of connections that the ACE discarded because the rservers where at max connections through the NAT Pool. |
| NAT bandwidth-rate-limit drop-count | Number of connections that the ACE discarded because the allocated bandwidth was exceeded through the NAT Pool. |
| Virtual IP (VIP) | The VIP address associated with this class. The VIP is specified by VIP, Protocol and Port. There can be multiple VIPs associated with each class. |
| L7 Policy | Name of the Layer 7 policy map associated with the service policy. |
| VIP Route Metric | Specifies the distance metric for the route as specified with the loadbalance vip advertise command. The ACE writes the value configured to its routing table. Possible values are integers from 1 to 254. |
| VIP Route Advertise | Operational state of the loadbalance vip advertise command: ENABLED or DISABLED. This command is used with Route Health Injection (RHI) to allow the ACE to advertise the availability of a VIP address throughout the network. |
| VIP ICMP Reply | Operational state of the loadbalance vip icmp-reply command. Possible states are: ENABLED, DISABLED, ENABLED-WHEN-ACTIVE, or ENABLED-WHEN-PRIMARY-SF-UP. |
| VIP State | Operational state of the virtual server: INSERVICE or OUTOFSERVICE. |
| Curr Conns | Number of active connections to the VIP. |
| Dropped Conns | Number of connections that the ACE discarded to this VIP. This is incremented whenever a connection hitting that VIP gets dropped/rejected. Common reasons that the connection hitting the VIP will be dropped are:
The counter itself is cumulative and that the value could be made up of entries from the following counters:
|
| Client Pkt Count | Number of packets received from the client side network to the VIP. |
| Server Pkt Count | Number of packets received from the server side network to the VIP. |
| Hit Count | Number of times a connection was established with this VIP. |
| Client Byte Count | Number of bytes received from the client side network to the VIP. |
| Server Byte Count | Number of bytes received from the server side network to the VIP. |
| Conn-rate-limit drop-count | Number of connections that the ACE discarded to this VIP because the rservers where at max connections |
| Bandwidth-rate-limit drop-count | Number of connections that the ACE discarded to this VIP because the allocated bandwidth was exceeded. |
| Primary server farm | The name of the primary server farm. |
| Current state of the primary server farm | The serverfarm can be either operational UP or DOWN. |
| Hit Count (Primary Server Farm) | Cumulative number of connections to the primary server farm. |
| Dropped Conns (Primary Server Farm) | Number of attempted connections to the primary server farm that the ACE discarded. |
| Backup server farm | The name of the backup server farm. |
| Current state of the backup server farm | The backup serverfarm can be either operational UP or DOWN. |
| Hit Count (Backup Server Farm) | Cumulative number of connections to the primary server farm. |
| Dropped Conns (Backup Server Farm) | Number of attempted connections to the primary server farm that the ACE discarded. |
| Parameter-map(s) | The name of any configured parameter maps which have been associated with this class. |
show stats loadbalance
Displays statistics related to load balancing activities of both np1 and np2. That is, it represents the sum of the values shown in show np 1 me-stats -slb and show np 2 me-stats -slb.
Sample Output
ACE/Admin# show stats loadbalance +------------------------------------------+ +------- Loadbalance statistics -----------+ +------------------------------------------+ Total version mismatch : 0 Total Layer4 decisions : 0 Total Layer4 rejections : 0 Total Layer7 decisions : 0 Total Layer7 rejections : 0 Total Layer4 LB policy misses : 0 Total Layer7 LB policy misses : 0 Total times rserver was unavailable : 0 Total ACL denied : 0 Total IDMap Lookup Failures : 0
Notes
| Field | Description |
| Total version mismatch | Vserver version on the proxy info or in the vserver state does not match that expected for the L7 connection. This occurs when new (reconfigured) vserver info is passed to the dataplane. LB is unable to return a decision, and the connection is close or not formed. |
| Total Layer4 decisions | LB loadBalanced, stuck, or forwarded an L4 connection. |
| Total Layer4 rejections | LB rejected an L4 connection. This is often in conjunction with other counters telling why. |
| Total Layer7 decisions | LB loadBalanced, stuck, or forwarded an L7 connection. |
| Total Layer7 rejections | LB rejected an L7 decision for any of a multitude of reasons, including:
For RADIUS traffic, if the client-side UDP connection and resulting pmap entry is removed before the RADIUS response is received by the ACE, ACE will not know where to forward the response. It therefore drops the packet and increments this counter. |
| Total Layer4 LB policy misses | LB was unable to find a policy which satisfies all the conditions of the L7 connection. |
| Total Layer7 LB policy misses | LB was unable to find a policy which satisfies all the conditions of the L7 connection. |
| Total times rserver was unavailable | Policy and vserver were OK, but there was no acceptable rserver available for LB to send the connection to. |
| Total ACL denied | The configured policy associated with this connection indicates it should be dropped. This can occur for either an L4 or an L7 policy. |
| Total IDMap Lookup Failures | Fault Tolerant (FT) ID MAP lookup failures. |
show stats crypto client
Sample Output
ACE30001/Admin# show stats crypto client SSL Client Statistics: ------------------ SSL alert CLOSE_NOTIFY rcvd: 0 SSL alert UNEXPECTED_MSG rcvd: 0 SSL alert BAD_RECORD_MAC rcvd: 0 SSL alert DECRYPTION_FAILED rcvd: 0 SSL alert RECORD_OVERFLOW rcvd: 0 SSL alert DECOMPRESSION_FAILED rcvd: 0 SSL alert HANDSHAKE_FAILED rcvd: 0 SSL alert NO_CERTIFICATE rcvd: 0 SSL alert BAD_CERTIFICATE rcvd: 0 SSL alert UNSUPPORTED_CERTIFICATE rcvd: 0 SSL alert CERTIFICATE_REVOKED rcvd: 0 SSL alert CERTIFICATE_EXPIRED rcvd: 0 SSL alert CERTIFICATE_UNKNOWN rcvd: 0 SSL alert ILLEGAL_PARAMETER rcvd: 0 SSL alert UNKNOWN_CA rcvd: 0 SSL alert ACCESS_DENIED rcvd: 0 SSL alert DECODE_ERROR rcvd: 0 SSL alert DECRYPT_ERROR rcvd: 0 SSL alert EXPORT_RESTRICTION rcvd: 0 SSL alert PROTOCOL_VERSION rcvd: 0 SSL alert INSUFFICIENT_SECURITY rcvd: 0 SSL alert INTERNAL_ERROR rcvd: 0 SSL alert USER_CANCELED rcvd: 0 SSL alert NO_RENEGOTIATION rcvd: 0 SSL alert CLOSE_NOTIFY sent: 0 SSL alert UNEXPECTED_MSG sent: 0 SSL alert BAD_RECORD_MAC sent: 0 SSL alert DECRYPTION_FAILED sent: 0 SSL alert RECORD_OVERFLOW sent: 0 SSL alert DECOMPRESSION_FAILED sent: 0 SSL alert HANDSHAKE_FAILED sent: 0 SSL alert NO_CERTIFICATE sent: 0 SSL alert BAD_CERTIFICATE sent: 0 SSL alert UNSUPPORTED_CERTIFICATE sent: 0 SSL alert CERTIFICATE_REVOKED sent: 0 SSL alert CERTIFICATE_EXPIRED sent: 0 SSL alert CERTIFICATE_UNKNOWN sent: 0 SSL alert ILLEGAL_PARAMETER sent: 0 SSL alert UNKNOWN_CA sent: 0 SSL alert ACCESS_DENIED sent: 0 SSL alert DECODE_ERROR sent: 0 SSL alert DECRYPT_ERROR sent: 0 SSL alert EXPORT_RESTRICTION sent: 0 SSL alert PROTOCOL_VERSION sent: 0 SSL alert INSUFFICIENT_SECURITY sent: 0 SSL alert INTERNAL_ERROR sent: 0 SSL alert USER_CANCELED sent: 0 SSL alert NO_RENEGOTIATION sent: 0 SSLv2 client hello received: 0 SSLv3 client hello received: 0 TLSv1 client hello received: 0 SSLv3 negotiated protocol: 0 TLSv1 negotiated protocol: 19855 SSLv3 full handshakes: 0 SSLv3 resumed handshakes: 0 Cipher sslv3_rsa_rc4_128_md5: 0 Cipher sslv3_rsa_rc4_128_sha: 0 Cipher sslv3_rsa_des_cbc_sha: 0 Cipher sslv3_rsa_3des_ede_cbc_sha: 0 Cipher sslv3_rsa_exp_rc4_40_md5: 0 Cipher sslv3_rsa_exp_des40_cbc_sha: 0 Cipher sslv3_rsa_exp1024_rc4_56_md5: 0 Cipher sslv3_rsa_exp1024_des_cbc_sha: 0 Cipher sslv3_rsa_exp1024_rc4_56_sha: 0 Cipher sslv3_rsa_aes_128_cbc_sha: 0 Cipher sslv3_rsa_aes_256_cbc_sha: 0 TLSv1 full handshakes: 19855 TLSv1 resumed handshakes: 0 Cipher tlsv1_rsa_rc4_128_md5: 19855 Cipher tlsv1_rsa_rc4_128_sha: 0 Cipher tlsv1_rsa_des_cbc_sha: 0 Cipher tlsv1_rsa_3des_ede_cbc_sha: 0 Cipher tlsv1_rsa_exp_rc4_40_md5: 0 Cipher tlsv1_rsa_exp_des40_cbc_sha: 0 Cipher tlsv1_rsa_exp1024_rc4_56_md5: 0 Cipher tlsv1_rsa_exp1024_des_cbc_sha: 0 Cipher tlsv1_rsa_exp1024_rc4_56_sha: 0 Cipher tlsv1_rsa_aes_128_cbc_sha: 0 Cipher tlsv1_rsa_aes_256_cbc_sha: 0 Total SSL client authentications: 0 Failed SSL client authentications: 0 SSL client authentication cache hits: 19841 SSL static CRL lookups: 0 SSL best effort CRL lookups: 0 SSL CRL lookup cache hits: 0 SSL revoked certificates: 0 Total SSL server authentications: 19855 Failed SSL server authentications: 0 Internal error: 0 Handshake FlushRX operations: 0 Handshake FlushTX operations: 0 Xscale messages rcvd from ME: 99275 Xscale messages sent to ME: 119130 Finish msg split across ssl recs: 0 Fasttx msg ring full: 0 SSL_ME tx msg ring full: 0 N2 encrypt_record: 0 N2 decrypt_record: 19855 N2 random: 39710 N2 handshake_hash: 19855 N2 hash: 0 N2 gpop_master: 19855 N2 gpop_import_master_secret: 0 N2 gpop_pkcs1v15enc: 19855 N2 gpop_pkcs1v15enc_crt: 0 N2 gpop_finish: 19855 N2 gpop_verify: 0 N2 gpop_pkcs1v15dec: 0 N2 gpop_pkcs1v15dec_crt: 0 N2 rsa_server_full: 0 N2 resume: 0
Notes
See show stats crypto server for a description of these counters.
show stats crypto server
Sample Output
ACE30001/Admin# show stats crypto server SSL Server Statistics: ------------------ SSL alert CLOSE_NOTIFY rcvd: 0 SSL alert UNEXPECTED_MSG rcvd: 0 SSL alert BAD_RECORD_MAC rcvd: 0 SSL alert DECRYPTION_FAILED rcvd: 0 SSL alert RECORD_OVERFLOW rcvd: 0 SSL alert DECOMPRESSION_FAILED rcvd: 0 SSL alert HANDSHAKE_FAILED rcvd: 0 SSL alert NO_CERTIFICATE rcvd: 0 SSL alert BAD_CERTIFICATE rcvd: 0 SSL alert UNSUPPORTED_CERTIFICATE rcvd: 0 SSL alert CERTIFICATE_REVOKED rcvd: 0 SSL alert CERTIFICATE_EXPIRED rcvd: 0 SSL alert CERTIFICATE_UNKNOWN rcvd: 0 SSL alert ILLEGAL_PARAMETER rcvd: 0 SSL alert UNKNOWN_CA rcvd: 0 SSL alert ACCESS_DENIED rcvd: 0 SSL alert DECODE_ERROR rcvd: 0 SSL alert DECRYPT_ERROR rcvd: 0 SSL alert EXPORT_RESTRICTION rcvd: 0 SSL alert PROTOCOL_VERSION rcvd: 0 SSL alert INSUFFICIENT_SECURITY rcvd: 0 SSL alert INTERNAL_ERROR rcvd: 0 SSL alert USER_CANCELED rcvd: 0 SSL alert NO_RENEGOTIATION rcvd: 0 SSL alert CLOSE_NOTIFY sent: 0 SSL alert UNEXPECTED_MSG sent: 0 SSL alert BAD_RECORD_MAC sent: 0 SSL alert DECRYPTION_FAILED sent: 0 SSL alert RECORD_OVERFLOW sent: 0 SSL alert DECOMPRESSION_FAILED sent: 0 SSL alert HANDSHAKE_FAILED sent: 0 SSL alert NO_CERTIFICATE sent: 0 SSL alert BAD_CERTIFICATE sent: 0 SSL alert UNSUPPORTED_CERTIFICATE sent: 0 SSL alert CERTIFICATE_REVOKED sent: 0 SSL alert CERTIFICATE_EXPIRED sent: 0 SSL alert CERTIFICATE_UNKNOWN sent: 0 SSL alert ILLEGAL_PARAMETER sent: 0 SSL alert UNKNOWN_CA sent: 0 SSL alert ACCESS_DENIED sent: 0 SSL alert DECODE_ERROR sent: 0 SSL alert DECRYPT_ERROR sent: 0 SSL alert EXPORT_RESTRICTION sent: 0 SSL alert PROTOCOL_VERSION sent: 0 SSL alert INSUFFICIENT_SECURITY sent: 0 SSL alert INTERNAL_ERROR sent: 0 SSL alert USER_CANCELED sent: 0 SSL alert NO_RENEGOTIATION sent: 0 SSLv2 client hello received: 0 SSLv3 client hello received: 0 TLSv1 client hello received: 0 SSLv3 negotiated protocol: 0 TLSv1 negotiated protocol: 0 SSLv3 full handshakes: 0 SSLv3 resumed handshakes: 0 Cipher sslv3_rsa_rc4_128_md5: 0 Cipher sslv3_rsa_rc4_128_sha: 0 Cipher sslv3_rsa_des_cbc_sha: 0 Cipher sslv3_rsa_3des_ede_cbc_sha: 0 Cipher sslv3_rsa_exp_rc4_40_md5: 0 Cipher sslv3_rsa_exp_des40_cbc_sha: 0 Cipher sslv3_rsa_exp1024_rc4_56_md5: 0 Cipher sslv3_rsa_exp1024_des_cbc_sha: 0 Cipher sslv3_rsa_exp1024_rc4_56_sha: 0 Cipher sslv3_rsa_aes_128_cbc_sha: 0 Cipher sslv3_rsa_aes_256_cbc_sha: 0 TLSv1 full handshakes: 0 TLSv1 resumed handshakes: 0 Cipher tlsv1_rsa_rc4_128_md5: 0 Cipher tlsv1_rsa_rc4_128_sha: 0 Cipher tlsv1_rsa_des_cbc_sha: 0 Cipher tlsv1_rsa_3des_ede_cbc_sha: 0 Cipher tlsv1_rsa_exp_rc4_40_md5: 0 Cipher tlsv1_rsa_exp_des40_cbc_sha: 0 Cipher tlsv1_rsa_exp1024_rc4_56_md5: 0 Cipher tlsv1_rsa_exp1024_des_cbc_sha: 0 Cipher tlsv1_rsa_exp1024_rc4_56_sha: 0 Cipher tlsv1_rsa_aes_128_cbc_sha: 0 Cipher tlsv1_rsa_aes_256_cbc_sha: 0 Total SSL client authentications: 0 Failed SSL client authentications: 0 SSL client authentication cache hits: 0 SSL static CRL lookups: 0 SSL best effort CRL lookups: 0 SSL CRL lookup cache hits: 0 SSL revoked certificates: 0 Total SSL server authentications: 0 Failed SSL server authentications: 0 Internal error: 0 Handshake FlushRX operations: 0 Handshake FlushTX operations: 0 Xscale messages rcvd from ME: 99285 Xscale messages sent to ME: 119142 Finish msg split across ssl recs: 0 Fasttx msg ring full: 0 SSL_ME tx msg ring full: 0 N2 encrypt_record: 0 N2 decrypt_record: 19857 N2 random: 39714 N2 handshake_hash: 19857 N2 hash: 0 N2 gpop_master: 19857 N2 gpop_import_master_secret: 0 N2 gpop_pkcs1v15enc: 19857 N2 gpop_pkcs1v15enc_crt: 0 N2 gpop_finish: 19857 N2 gpop_verify: 0 N2 gpop_pkcs1v15dec: 0 N2 gpop_pkcs1v15dec_crt: 0 N2 rsa_server_full: 0 N2 resume: 0
Notes
| Field | Description |
| SSL alert [type] rcvd/sent | Number of times that the particular SSL alert message was received or sent. |
| SSLv2/v3 client hello received | Number of ClientHello message received. |
| SSLv3/TLSv1 negotiated protocol | Number of the times that the version used in the connection. |
| SSLv3 full handshakes | Number of handshakes completed without errors. |
| SSLv3 resumed handshakes | Number of handshakes resumed by using a session ID. |
| Cipher sslv3... | Number of times that the cipher suite is used in the connection. |
| TLSv1 full handshakes | Number of handshakes completed without errors. |
| TLSv1 resumed handshakes | Number of handshakes resumed by using a session ID. |
| Cipher tlsv1... | Number of times that the cipher suite is used in the connection. |
| Total SSL client authentications | Number of authenticated client connections. This field increments only when displaying server statistics. |
| Failed SSL client authentications | Number of client connections that failed authentication. This field increments only when displaying server statistics. |
| SSL client authentication cache hits | Number of times that an authenticated client reconnects and a cache entry is found. This field increments only when displaying server statistics. |
| SSL static CRL lookups | Number of lookups against a statically defined CRL. |
| SSL best effort CRL lookups | Number of lookups using the best effort. |
| SSL CRL lookup cache hits | Number of CRL lookups where the cache result was used. |
| SSL revoked certificates | Number of revoked certificates encountered. |
| SSL CRL download failed | Number of CRL downloads that failed. |
| Total SSL server authentications | Number of server certificate authentications that the ACE attempted to perform. This field increments only when displaying client statistics. |
| Failed SSL server authentications | Number of server certificate authentications that failed. This field increments only when displaying client statistics. |
| Handshake FlushRX/TX operations | Number of times that the SSL handshake finished. |
| Xscale messages rcvd/sent for ME | Number of messages passed between the SSL processors during the SSL handshake. |
| Xscale rcvd abort msg before hdshk | Number of times that the SSL handshake was aborted. |
| Finish msg split across ssl recs | Number of times that the SSL Finished message was split by the client into multiple SSL records. |
| SSLv3 client hello received | Number of SSL V3 client hello messages received. |
| SSLv3 negotiated protocol | Number of times the the SSL v3 protocol was selected in the handshake. |
| Cipher sslv3_rsa_rc4_128_md5 | Number of times this cipher suite was selected. |
| N2 rsa_server_full | N2 rsa_server_full is called as a normal part of the SSL handshake. |
Notes
If "SSLv3 full handshakes" is less than rsa_server_full, then the difference is the number of connections that did not complete the SSL handshake. This should be close to the value of "Handshake failure alert".
The sum of "SSLv3 client hello received" and "TLSv1 client hello received", less "N2 rsa_server_full" gives you the number of handshakes that failed prior to getting to "N2 rsa_server_full".
See also show np 1 me-stats -snitrox and show np 1 me-stats -scrypto.
show stats loadbalance radius
Sample Output
slot3/radius# show stats loadbalance radius +-------------------------------------------------+ +--------- Radius Loadbalance statistics ---------+ +-------------------------------------------------+ Total requests received : 0 Total responses received : 0 Total retry packets received : 0 Total header parse results received : 0 Total body parse results received : 0 Total data parse results received : 0 Total packets sent out : 0 Total sessions allocated : 0 Total sessions deleted : 0 Total username sticky added : 0 Total calling-station sticky added : 0 Total framed-ip sticky added : 0 Total end-user packet sticky success : 0 Total end-user packet sticky failure : 0 Total Acct-On/Off requests received : 0 Total Acct-On/Off responses received : 0 Total Acct-On/Off with no rules : 0 Total Acct-On/Off req processing done: 0 Total NULL packet received errors : 0 Total parse errors : 0
Notes
While most of the counters are self-explanatory, note the following points regarding several of the counters:
| Field | Description |
| Total proxy mapper errors | This counter increments if there is no Radius header in the Radius message or if in other ways the message is malformed in such as way as to affect the header. |
| Total memory allocation failures | This counter increments when a pmap entry cannot be allocated for the Radius transaction. There are 32K pmap entries for each IXP. A pmap entry is created for each Radius transaction. Note that there can be multiple radius transactions per UDP connection. If this stat is incrementing, then radius messages are being dropped, and the recommended workaround is to reduce the UDP inactivity timeout. (Make it UDP ports 1812 and 1813 for minimal impact on other connections).
WARNING: If the timeout is too short, the client-side connection and resulting pmap entry might be removed before the radius response message is received by the ACE. In this case, the response is dropped and "Total Layer7 rejections" counter will be incremented under show stats loadbalance radius. |
| Total end-user packet sticky failure | This counter is incremented every time a Radius end-user packet is received which does not hit any sticky entry. In this case the pkt will be forwarded (routed) to the server and this counter will be incremented. To help clarify, this would happen with a config such as:
sticky radius framed-ip GRP1
replicate sticky
serverfarm sf1
class-map match-all catch_all_vip
2 match virtual-address 0.0.0.0 0.0.0.0 any
policy-map type loadbalance first-match L7_P
class class-default
sticky-serverfarm GRP1
policy-map multi-match RAD_L7_VIP
class RAD_L7-VIP
loadbalance vip inservice
loadbalance policy RAD_L7_pmap
loadbalance vip icmp-reply
class catch_all_vip
loadbalance vip inservice
loadbalance policy L7_P
loadbalance vip icmp-reply
interface vlan 440
ip address 44.44.44.149 255.255.255.0
service-policy input RAD_L7_VIP
no shutdown
In this scenario, if there were no Radius FIP sticky entry, and then an end-user sent a packet to a server that matched the catch_all_vip (for example using directly the server IP address, not the VIP), it will match the catch_all_vip and the L7_P. A sticky lookup will be done but no sticky entry is found. Then this counter is incremented. |
| Total Valid Sticky Key | This will increase every time an access-accept hits a pmap entry that has a valid sticky key in it (which basically means that the previous access-request was a retransmission). |
show stats probe
Displays probe-related statistics.
Sample Output
ace-1/Admin# show stats probe +------------------------------------------+ +----------- Probe statistics -------------+ +------------------------------------------+ ----- icmp probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- tcp probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- udp probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- http probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- https probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- ftp probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- telnet probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- smtp probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- pop probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- dns probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- echo probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- imap probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- radius probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- finger probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0 ----- script probe ---- Total probes sent : 0 Total send failures : 0 Total probes passed : 0 Total probes failed : 0 Total connect errors : 0 Total conns refused : 0 Total RST received : 0 Total open timeouts : 0 Total receive timeout : 0
Notes
For a description of the fields, see the "Displaying Probe Information" chapter of the Application Control Engine Module Server Load-Balancing Configuration Guide.
show syn-cookie
Displays the SYN cookie-related statistics for each configured interface. A TCP SYN cookie is an initial sequence number calculated in response to a SYN request from a client and inserted in the SYN-ACK response. It provides an authentication mechanism of sorts for preventing SYN floods from a rogue client.
Sample Output
ACE30001/Admin# show syn-cookie
Interface vlan23
Configured TCP Embryonic Connection Limit: 0
Current number of Embryonic Connections: 0
Number of TCP Syns Intercepted by SYN COOKIE: 0
Number of TCP Acks Successfully Processed by SYN COOKIE: 0
Failed Number of TCP Acks Processed by SYN COOKIE: 0
Interface vlan101
Configured TCP Embryonic Connection Limit: 0
Current number of Embryonic Connections: 0
Number of TCP Syns Intercepted by SYN COOKIE: 0
Number of TCP Acks Successfully Processed by SYN COOKIE: 0
Failed Number of TCP Acks Processed by SYN COOKIE: 0
Interface vlan102
Configured TCP Embryonic Connection Limit: 0
Current number of Embryonic Connections: 0
Number of TCP Syns Intercepted by SYN COOKIE: 0
Number of TCP Acks Successfully Processed by SYN COOKIE: 0
Failed Number of TCP Acks Processed by SYN COOKIE: 0
show system internal mts buffers details
Provides detailed information on MTS buffers. MTS Buffers are used for internal communications between processes and threads with the ACE. In general a buffer is allocated, sent, processed and freed quickly.
Sample Output
ACE30001/Admin# show system internal mts buffers details Node/Sap/queue Age(ms) SrcNode SrcSAP DstNode DstSAP OPC MsgId MsgSize sup/1106/nper 26497 0x601 1085 0x601 1106 4001 927650 300 sup/1085/nper 26497 0x601 1085 0x601 1106 4001 927650 300
Notes
It would be normal to see a handful of buffers in the output; however, it is most likely a problem when many are observed or the "Age" is very large. This would indicate that the buffer has been leaked. Once the ACE runs low on these MTS buffer, the ACE can reboot without a core, possibly making the Control Plane (CP) become unreachable which would lead to management access problems or other unusual issues.
In general, if there are a lot of buffers on this output with a large age you should notify Cisco TAC.
show system internal mts memory
Sample Output
ACE30001/Admin# show system internal mts memory mts buffer manager statistics shared memory pool at c111b000-c211b000 (size: 16777216 bytes) request_hi: 41476 request_lo: 0 mem_in_use: 0 mem_free: 16777216 num_buffers: 0 mem_in_use_hi: 131328 num_allocs: 1655510 num_frees: 1655510 num_successes: 1655510 num_failures: 0 frag_waste: 0 frag_high: 55308
Notes
| Field | Description |
| request_hi | Highest sized memory block requested |
| request_lo | Lowest sized memory block requested |
| mem_in_use | Total memory allocated |
| mem_free | Total memory free |
| num_buffers | Total number of buffers allocated currently |
| mem_in_use_hi | Highest level of memory in use ever |
| num_alloc | Number of allocation requests |
| num_frees | Number of free requests |
| num_successes | Number of successful alloc requests |
| num_failures | Number of failures |
| frag_waste | Space currently wasted in internal fragmentation |
| frag_high | Highest level of fragmentation ever reached |
show system internal sysmgr service all details
Displays all the available services that have started on the ACE. Some examples of a service (or more accurately, "daemon") that run on ACE are: TACACS or RADIUS.
Sample Output
switch/Admin# show system internal sysmgr service all details
Service "Tacacs Daemon" ("tacacs", 2):
UUID = 0xB6, PID = 923, SAP = 112
State: SRV_STATE_HANDSHAKED (entered at time Fri Dec 28 20:12:25 2007).
Restart count: 1
Time of last restart: Fri Dec 28 20:12:23 2007.
The service never crashed since the last reboot.
Notes
The fields of interest are:
- "Restart count" — This should always be a value of one on a customer ACE. In production environments if a service were to crash it would bring down ACE and generate a core dump.
- "Time of last restart" — On a customer ACE this will be the time of the last core dump for this server if it has ever caused the ACE to crash and reboot. The reboot reason will be on the second line.
A process core will contain this output for the service that cored. On a running ACE this command would not be of interest.
show system kcache
Displays information on the cache used by various processes.
Sample Output
ACE30001/Admin# show system kcache slabinfo - version: 1.1 (SMP) kmem_cache 80 80 244 5 5 1 : 252 126 ipcp_pp_req_cache 118 118 64 2 2 1 : 252 126 ipcp_qentry_cache 213 339 32 3 3 1 : 252 126 mts-qentry-cache 213 339 32 3 3 1 : 252 126 mts-sapinfo-cache 14 14 1152 2 2 2 : 60 30 tl_mts_cache 339 339 32 3 3 1 : 252 126 scp_mts_cache 328 404 16 2 2 1 : 252 126 tcp_tw_bucket 280 280 96 7 7 1 : 252 126 tcp_bind_bucket 213 339 32 3 3 1 : 252 126 tcp_open_request 118 118 64 2 2 1 : 252 126 inet_peer_cache 59 59 64 1 1 1 : 252 126 ip_fib_hash 226 226 32 2 2 1 : 252 126 ip_dst_cache 260 260 192 13 13 1 : 252 126 arp_cache 30 30 128 1 1 1 : 252 126 blkdev_requests 160 160 96 4 4 1 : 252 126 nfs_write_data 0 0 416 0 0 1 : 124 62 nfs_read_data 0 0 384 0 0 1 : 124 62 nfs_page 0 0 96 0 0 1 : 252 126 journal_head 186 312 48 4 4 1 : 252 126 revoke_table 252 253 12 1 1 1 : 252 126 revoke_record 226 226 32 2 2 1 : 252 126 dnotify cache 0 0 20 0 0 1 : 252 126 file lock cache 120 120 96 3 3 1 : 252 126 fasync cache 0 0 16 0 0 1 : 252 126 uid_cache 226 226 32 2 2 1 : 252 126 skbuff_head_cache 1474 1600 192 80 80 1 : 252 126 sock 496 496 896 124 124 1 : 124 62 sigqueue 261 261 132 9 9 1 : 252 126 cdev_cache 118 118 64 2 2 1 : 252 126 bdev_cache 118 118 64 2 2 1 : 252 126 mnt_cache 118 118 64 2 2 1 : 252 126 inode_cache 2254 2254 512 322 322 1 : 124 62 dentry_cache 2880 2880 128 96 96 1 : 252 126 filp 3150 3150 128 105 105 1 : 252 126 names_cache 22 22 4096 22 22 1 : 60 30 buffer_head 18698 28400 96 468 710 1 : 252 126 mm_struct 312 312 160 13 13 1 : 252 126 vm_area_struct 8788 9040 96 222 226 1 : 252 126 fs_cache 354 354 64 6 6 1 : 252 126 files_cache 198 198 416 22 22 1 : 124 62 signal_act 80 87 4128 27 29 4 : 0 0 size-131072(DMA) 0 0 131072 0 0 32 : 0 0 size-131072 3 4 131072 3 4 32 : 0 0 size-65536(DMA) 0 0 65536 0 0 16 : 0 0 size-65536 20 20 65536 20 20 16 : 0 0 size-32768(DMA) 0 0 32768 0 0 8 : 0 0 size-32768 15 15 32768 15 15 8 : 0 0 size-16384(DMA) 0 0 16384 0 0 4 : 0 0 size-16384 12 12 16384 12 12 4 : 0 0 size-8192(DMA) 0 0 8192 0 0 2 : 0 0 size-8192 22 25 8192 22 25 2 : 0 0 size-4096(DMA) 0 0 4096 0 0 1 : 60 30 size-4096 31 31 4096 31 31 1 : 60 30 size-2048(DMA) 0 0 2048 0 0 1 : 60 30 size-2048 1210 1210 2048 605 605 1 : 60 30 size-1024(DMA) 0 0 1024 0 0 1 : 124 62 size-1024 462 524 1024 118 131 1 : 124 62 size-512(DMA) 0 0 512 0 0 1 : 124 62 size-512 1696 1696 512 212 212 1 : 124 62 size-256(DMA) 0 0 256 0 0 1 : 252 126 size-256 324 450 256 26 30 1 : 252 126 size-128(DMA) 0 0 128 0 0 1 : 252 126 size-128 1890 1890 128 63 63 1 : 252 126 size-64(DMA) 0 0 64 0 0 1 : 252 126 size-64 641 767 64 12 13 1 : 252 126 size-32(DMA) 0 0 32 0 0 1 : 252 126 size-32 2938 2938 32 26 26 1 : 252 126
Notes
This is the same as the linux command 'cat /proc/slabinfo'.
show system kmem
Displays system kernel memory statistics. Using this command to monitor memory use over the course of several hours can help identify memory leaks in the system.
Sample Output
ACE30001/Admin# show system kmem
total: used: free: shared: buffers: cached:
Mem: 847564800 591683584 255881216 0 3371008 168300544
Swap: 0 0 0
MemTotal: 827700 kB
MemFree: 249884 kB
MemShared: 0 kB
Buffers: 3292 kB
Cached: 164356 kB
SwapCached: 0 kB
Active: 4436 kB <<<
Inactive: 514964 kB
HighTotal: 655360 kB
HighFree: 166460 kB
LowTotal: 172340 kB
LowFree: 83424 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Committed_AS: 948556 kB
Notes
This output displays is the same information that is displayed via the linux command cat /proc/meminfo.
Memory values are in units of kBytes.
| Field | Description |
| MemTotal | Total usable RAM: physical memory minus reserved memory and the kernel binary code. |
| MemFree | The amount of physical RAM left unused by the system. |
| Buffers | The amount of physical RAM used for file buffers. |
| Cached | The amount of physical RAM used as cache memory. Memory in the pagecache (diskcache) minus SwapCache. |
| SwapCache | The amount of Swap used as cache memory. Memory that once was swapped out, is swapped back in, but is still in the swapfile. |
| Active | The total amount of buffer or page cache memory, that is active. This part of the memory is used recently and usually not reclaimed unless absolutely necessary. |
| Inactive | The total amount of buffer or page cache memory that are free and available. This is memory that has not been recently used and can be reclaimed for other purposes by the paging algorithm. |
| HighTotal | The total amount of memory in the high region. This can vary based on the type of kernel used. Kernel uses indirect tricks to access the high memory region. Data cache can go in this memory region. |
| LowTotal | The total amount of non-highmem memory. |
| LowFree | The amount of free memory of the low memory region. This is the memory the kernel can address directly. All kernel datastructures need to go into low memory. |
| SwapTotal | Total amount of physical swap memory. |
| SwapFree | Total amount of swap memory free. |
| Dirty | The total amount of memory waiting to be written back to the disk. |
| Writeback | The total amount of memory actively being written back to the disk. |
| Committed_AS | An estimate of how much RAM you would need to make a 99.99% guarantee that there never is Out of Memory (OOM) for this workload. Normally the kernel will overcommit memory. This parameter represents the worst case scenario value and also includes swap memory. |
show system kmemtrack
Enables you to track memory allocations and frees in the kernel loadable modules (drivers).
Sample Output
ACE30001/Admin# show system kmemtrack Kernel Module Memory Tracking ------------------------------------------------------------- Module kmalloc kfree diff klm_cpu_util 00000008 00000008 00000000 klm_ipcp 00001811 00001811 00000000 n2_drv 00000339 00000227 00000112 klm_bf_nvram 00000016 00000016 00000000 klm_idprom 00000071 00000071 00000000 klm_encdec 00000011 00000000 00000011 klm_modlock 00000001 00000001 00000000 klm_sysmgr-hb 00000002 00000000 00000002 klm_utaker 00006656 00006656 00000000 klm_mts 00248775 00247519 00001256 klm_scp 00000001 00000000 00000001 klm_tl 00119846 00119843 00000003 klm_sdwrap 00001838 00000018 00001820 klm_resdrv 00000001 00000001 00000000 klm_klib 00000004 00000000 00000004
Notes
The "diff" field is useful if you suspect that there is a memory leak in the kernel. Not just the absolute value in the diff column but continuously increasing value across successive invocations of this command.
- show proc memory is for the user space processes
- show proc kmemtrack is for drivers alone
- show proc kmem is for whole system.
The operator should start with show system kmem to see if system is running low on memory. If it is one of the other two commands can shows who process or driver is taking up too much memory.
show system resources
Sample Output
ACE30001/Admin# show system resources
Load average: 1 minute: 0.06 5 minutes: 0.09 15 minutes: 0.09
Processes : 116 total, 3 running
CPU states : 50.5% user, 2.0% kernel, 47.5% idle
Memory usage: 827700K total, 587560K used, 240140K free
3332K buffers, 173304K cache
Notes
- Load average: These stats correspond to the load averages from the linux "w" command
- Processes: Total number of processes - do 'show proc cpu' for details
- CPU states: CPU Percentages
- Memory usage: These stats correspond to the linux command cat /proc/meminfo
show system skbtrack
This command displays the socket buffer (network buffer, or skbuffs) allocations in the kernel loadable modules. Skbuffs are the buffers in which the OS kernel handles network packets. The packet is received by the network card, put into an skbuff and then passed to the network stack, which uses the skbuff.
Sample Output
ACE30001/Admin# show system skbtrack Kernel Module skbuff Tracking ------------------------------------------------------------- Module alloc free diff n2_drv 00001090 00000000 00001090 klm_pseudo 00397987 00397987 00000000 klm_mts 00779257 00779257 00000000
Notes
- Module: kernel loadable module (KLM) name
- alloc: network buffer allocations
- free: network buffer frees
- diff: memory allocated minus free
show xlate
Sample Output
ACE30001/Admin# show xlate TCP PAT from vlan32:10.25.49.131/14311 to vlan32:172.16.182.170/1304 NAT from vlan32:192.168.25.73 to vlan55:172.16.183.33 count:1
Notes
In the PAT line, the listed elements are as follows:
- The client IP is 10.25.49.131
- TCP port is 14311
- Client VLAN is 32
- Server VLAN is 32
- PAT address is 172.16.182.170
- PAT port = 1304
Similarly, in the NAT line, the client IP and VLAN is on the left and the server IP and VLAN is on the right.
