Cisco WAAS Troubleshooting Guide for Release 4.1.3 and Later -- Troubleshooting the Generic AO
This article describes how to troubleshoot the Generic AO.
Generic Accelerator Troubleshooting
The Generic accelerator optimizes traffic that is pushed down from the other AOs because they cannot optimize the traffic. The generic AO performs TFO optimization only. (DRE and LZ compression optimizations are performed by the SO-DRE component)
The generic AO receives connections under the following conditions:
- Failure case: An AO determines that it cannot handle the connection after sensing that the data is incomprehensible to it. For example, if the CIFS AO senses encrypted data or unauthenticated content, it will not be able to handle it and will push down the connection to the generic AO.
- Multiple protocol handling: For instance, the video AO could accept all connections that are related to multiple protocols like WMT, RTSP, and so on. However, the video AO currently provides only RTSP optimization, so it will not handle the connections that are related to other protocols and it will push down these connections to the generic AO.
Common scenarios in which connections are pushed down to the generic AO include the following conditions where there is connectivity that the AO does not understand or cannot optimize:
- Unauthenticated CIFS
- SMB-signed CIFS
- Encrypted MAPI
- Non-RTSP video
One way to check if the generic AO is being used is to look at statistics from the other AOs. For example, the CIFS AO reports connections that are pushed down to the generic AO as follows:
WAE674# sh stat accelerator cifs detail CIFS: Global Statistics ----------------- Time Accelerator was started: Tue Jul 14 11:55:09 2009 Time Statistics were Last Reset/Cleared: Thu Jul 16 04:16:35 2009 Total Handled Connections: 32 Total Optimized Connections: 1 Total Connections Handed-off with Compression Policies Unchanged: 24 <-----Pushed down to generic AO Total Dropped Connections: 0 Current Active Connections: 0 Current Pending Connections: 0 Maximum Active Connections: 4 Number of local reply generating requests: 3388 Number of remote reply generating requests: 415 The Average time to generate a local reply (msec): 25 Average time to receive remote reply (ms): 2147
You can also check connection statistics to see what optimizations are being applied to connections. In the show statistics connection output, a "G" indicates the connection was handled by the generic AO as follows:
WAE674# sh stat connection Current Active Optimized Flows: 2 Current Active Optimized TCP Plus Flows: 2 Current Active Optimized TCP Only Flows: 0 Current Active Optimized TCP Preposition Flows: 0 Current Active Auto-Discovery Flows: 5 Current Active Pass-Through Flows: 0 Historical Flows: 100 D:DRE,L:LZ,T:TCP Optimization, A:AOIM,C:CIFS,E:EPM,G:GENERIC,H:HTTP,M:MAPI,N:NFS,S:SSL,V:VIDEO ConnID Source IP:Port Dest IP:Port PeerID Accel 3722 10.10.10.10:2162 10.10.100.100:445 00:14:5e:84:24:5f TCDL 3924 10.10.10.10:2464 10.10.100.101:445 00:14:5e:84:24:5f TGDL <-----Look for "G"
If you take a closer look at the connection above, you will see that CIFS was configured but the generic AO was applied as follows:
WAE674# sh stat connection conn-id 3924 Connection Id: 3924 Peer Id: 00:14:5e:84:24:5f Connection Type: EXTERNAL CLIENT Start Time: Thu Jul 16 06:10:44 2009 Source IP Address: 10.10.10.10 Source Port Number: 2464 Destination IP Address: 10.10.100.101 Destination Port Number: 445 Application Name: WAFS Classifier Name: CIFS Map Name: basic Directed Mode: FALSE Preposition Flow: FALSE Policy Details: Configured: TCP_OPTIMIZE + DRE + LZ Derived: TCP_OPTIMIZE + DRE + LZ Peer: TCP_OPTIMIZE + DRE + LZ Negotiated: TCP_OPTIMIZE + DRE + LZ Applied: TCP_OPTIMIZE + DRE + LZ Accelerator Details: Configured: CIFS <-----CIFS configured Derived: CIFS Applied: GENERICAO <-----Generic applied Hist: CIFS
To view similar information from the Central Manager, choose the WAE device, then choose Monitor > Optimization > Connections Statistics. Connections handled by the generic AO look as follows:
- Figure 1. Connection Statistics Report with Generic
You can use the show statistics accelerator generic detail command to see more details about connections being handled by the generic AO as follows:
WAE# sh stat accelerator generic detail Generic: ------- Time elapsed since "clear statistics": 1days 18hr 25min 20sec Time Accelerator was started: Tue Jul 14 11:55:02 2009 Time Statistics were Last Reset/Cleared: Tue Jul 14 11:55:02 2009 Total Handled Connections: 366 Total Optimized Connections: 366 Total Connections Handed-off with Compression Policies Unchanged: 0 Total Dropped Connections: 0 Current Active Connections: 1 Current Pending Connections: 0 Maximum Active Connections: 2 . . . Global Generic AO connection statistics ======================================= Total number of connections handled: 366 <------------ Total number of active connections: 1 Total number of bytes transferred from client: 12055 Total number of bytes transferred from server: 12492 Global Generic AO connection error statistics ============================================= Source connection closed: 730 Destination connection closed: 0 Source connection aborted: 0 Destination connection aborted: 0 Source connection error: 0 Destination connection error: 0 Out of memory: 0 Kernel Queue abort error: 0
If you are seeing a large Total number of connections handled, some kind of configuration or communication error might be causing a large number of connections to be pushed down.
Generic AO Logging
The following log files are available for troubleshooting generic AO issues:
- Transaction log files: /local1/logs/tfo/working.log (and /local1/logs/tfo/tfo_log_*.txt)
- Debug log files: /local1/errorlog/genericao-errorlog.current (and genericao-errorlog.*)
For easier debugging, you should first set up an ACL to restrict packets to one host.
WAE674(config)# ip access-list extended 150 permit tcp host 10.10.10.10 any WAE674(config)# ip access-list extended 150 permit tcp any host 10.10.10.10
To enable transaction logging, use the transaction-logs configuration command as follows:
wae(config)# transaction-logs flow enable wae(config)# transaction-logs flow access-list 150
To set up and enable debug logging of the generic AO, use the following commands.
NOTE: Debug logging is CPU intensive and can generate a large amount of output. Use it judiciously and sparingly in a production environment.
You can enable detailed logging to the disk as follows:
WAE674(config)# logging disk enable WAE674(config)# logging disk priority detail
You can enable debug logging for connections in the ACL as follows:
WAE674# debug connection access-list 150
The options for generic AO debugging are as follows:
WAE674# debug accelerator generic ? all enable all GENERIC accelerator debugs connection enable GENERIC accelerator connection debugs misc enable GENERIC accelerator miscellaneous debugs shell enable GENERIC accelerator shell debugs stats enable GENERIC accelerator stats debugs
You can enable debug logging for generic AO connections and then display the end of the debug error log as follows:
WAE674# debug accelerator generic connection WAE674# type-tail errorlog/genericao-errorlog.current follow