Cisco WAAS Troubleshooting Guide for Release 4.1.3 and Later -- Troubleshooting the MAPI AO

From DocWiki

(Difference between revisions)
Jump to: navigation, search
m (1 revision)

Revision as of 03:24, 1 June 2011

This article describes how to troubleshoot the MAPI AO.


Guide Contents
Main Article
Understanding the WAAS Architecture and Traffic Flow
Preliminary WAAS Troubleshooting
Troubleshooting Optimization
Troubleshooting Application Acceleration
Troubleshooting the CIFS AO
Troubleshooting the HTTP AO
Troubleshooting the EPM AO
Troubleshooting the MAPI AO
Troubleshooting the NFS AO
Troubleshooting the SSL AO
Troubleshooting the Video AO
Troubleshooting the Generic AO
Troubleshooting Overload Conditions
Troubleshooting WCCP
Troubleshooting Disk and Hardware Problems
Troubleshooting Serial Inline Clusters
Troubleshooting vWAAS
Troubleshooting WAAS Express
Troubleshooting NAM Integration

Contents




MAPI Accelerator

The MAPI accelerator optimizes Microsoft Outlook Exchange e-mail traffic. Exchange uses the EMSMDB protocol, which is layered on MS-RPC, which in turn uses either TCP or HTTP (unsupported) as the low level transport.

The MAPI AO supports Microsoft Outlook 2000 through 2007 clients for both cached and noncached mode traffic. Secure connections that use message authentication (signing) or encryption are not accelerated by the MAPI AO. Such connections and connections from older clients are handed off to the generic AO for TFO optimizations. Additionally, Outlook Web Access (OWA) and Exchange-Exchange connections are not supported.

Note: Microsoft Outlook 2007 has encryption enabled by default. You must disable encryption to benefit from the MAPI application accelerator. In Outlook, choose Tools > E-mail Accounts, choose View or Change Existing E-mail Accounts, and then click Next. Choose the Exchange account, and then click Change. Click More Settings, and then click the Security tab. Uncheck the Encrypt data between Microsoft Office Outlook and Microsoft Exchange Server check box, as shown in Figure 1.

Alternatively, you can disable encryption for all users of an Exchange Server by using a Group Policy.

Figure 1. Disabling Encryption in Outlook 2007
Waast-outlookencrypt.png

In the following cases, the MAPI AO does not handle a connection:

  • Encrypted connection (handed off to the generic AO)
  • Unsupported client (handed off to the generic AO)
  • Unrecoverable parsing error. All TCP connections between the client and server service are disconnected. When the client reconnects, all connections are handed off to the generic AO.
  • Client attempts to establish a new association group on the connection when the WAE is overloaded.
  • Client establishes a connection when the WAE is overloaded and MAPI reserved connection resources are not available.

The Outlook client and server interact in a session over a group of TCP connections called an association group. Within an association group, object accesses can span across any connection and connections are dynamically created and released as needed. A client can have more than one association group open at the same time to different servers or the same server. (Public folders are deployed on different servers from the mail store.)

It is essential that all MAPI connections within an association group go through the same pair of WAEs in the branch and data center. If some connections within an association group do not go through the MAPI AO on these WAEs, the MAPI AO would not see the transactions performed on those connections and the connections are said to "escape" the association group. For this reason, the MAPI AO should not be deployed on serially clustered inline WAEs that form a high availability group.

The symptoms of MAPI connections that escape their WAE association group are Outlook error symptoms such as duplicate messages or Outlook stops responding.

During a TFO overload condition, new connections for an existing association group would be passed through and escape the MAPI AO, so the MAPI AO reserves a number of connection resources in advance to minimize the impact of an overload condition. For more details about reserved MAPI connections and their impact on device overload, see the section "MAPI Application Accelerator Reserved Connections Impact on Overload" in the Troubleshooting Overload Conditions article.

Verify the general AO configuration and status with the show accelerator and show license commands, as described in the Troubleshooting Application Acceleration article. The Enterprise license is required for MAPI accelerator operation and the EPM application accelerator must be enabled.

Next, verify the status specific to the MAPI AO by using the show accelerator mapi command, as shown in Figure 2. You want to see that the MAPI AO is Enabled, Running, and Registered, and that the connection limit is displayed. If the Config State is Enabled but the Operational State is Shutdown, it indicates a licensing problem.

Figure 2. Verifying the MAPI Accelerator Status
Waast-mapiaostatus.png

Use the show statistics accelerator epm command to verify that the EPM AO is functional. Check that the Total Handled Connections, Total Requests Successfully Parsed, and Total Responses Successfully Parsed counters increase when a client is started.

Use the show running-config command to verify that the MAPI and EPM traffic policies are properly configured. You want to see accelerate mapi for the Email-and-Messaging application action and you want to see the MS-EndPortMapper classifier and traffic policy defined, as follows:

WAE674# sh run | include mapi
 map adaptor EPM mapi
   name Email-and-Messaging All action optimize full accelerate mapi

WAE674# sh run | begin MS-EndPointMapper
...skipping
 classifier MS-EndPointMapper
    match dst port eq 135
 exit

WAE674# sh run | include MS-EndPointMapper
    classifier MS-EndPortMapper
      name Other classifier MS-EndPortMapper action optimize DRE no compression none accelerate MS-port-mapper

Use the show policy-engine application dynamic command to verify that dynamic match rules exist, as follows:

  • Look for a rule with User ID: EPM and Map Name: uuida4f1db00-ca47-1067-b31f-00dd010662da.
  • The Flows field indicates the total number of active connections to the Exchange service.
  • For each MAPI client you should see a separate entry with the User ID: MAPI.

Use the show statistics connection optimized mapi command to check that the WAAS device is establishing optimized MAPI connections. Verify that "M" appears in the Accel column for MAPI connections, which indicates that the MAPI AO was used, as follows:

WAE674# show stat conn opt mapi

Current Active Optimized Flows:                      2
   Current Active Optimized TCP Plus Flows:          1
   Current Active Optimized TCP Only Flows:          1
   Current Active Optimized TCP Preposition Flows:   0
Current Active Auto-Discovery Flows:                 0
Current Reserved Flows:                              12               <---------- Added in 4.1.5
Current Active Pass-Through Flows:                   0
Historical Flows:                                    161


D:DRE,L:LZ,T:TCP Optimization RR:Total Reduction Ratio
A:AOIM,C:CIFS,E:EPM,G:GENERIC,H:HTTP,M:MAPI,N:NFS,S:SSL,V:VIDEO

ConnID  Source IP:Port         Dest IP:Port        PeerID           Accel RR
342     10.56.94.101:4506    10.10.100.100:1456  0:1a:64:d3:2f:b8   TMDL  61.0%   <-----Look for "M"

Note: In version 4.1.5, the Current Reserved Flows counter was added in the output. This counter refers to the number of reserved MAPI connection resources on the WAE that are currently unused but set aside for future MAPI connections. For more details about reserved MAPI connections and their impact on device overload, see the section "MAPI Application Accelerator Reserved Connections Impact on Overload" in the Troubleshooting Overload Conditions article.

If you observe connections with "TGDL" in the Accel column, these connections were pushed down to the generic AO and optimized with transport optimizations only. If these are connections that you expected to be handled by the MAPI AO, it may be because they are encrypted MAPI connections. To check on the number of encrypted MAPI connections that have been requested, use the show statistics accelerator mapi command as follows:

wae# sh stat accel mapi

MAPI:
  Global Statistics
  -----------------
  Time Accelerator was started:                                      Thu Nov  5 19:45:19 2009
  Time Statistics were Last Reset/Cleared:                           Thu Nov  5 19:45:19 2009
  Total Handled Connections:                                         8615     
  Total Optimized Connections:                                       8614     
  Total Connections Handed-off with Compression Policies Unchanged:  0        
  Total Dropped Connections:                                         1        
  Current Active Connections:                                        20       
  Current Pending Connections:                                       0        
  Maximum Active Connections:                                        512      
  Number of Synch Get Buffer Requests:                               1052     
  Minimum Synch Get Buffer Size (bytes):                             31680    
  Maximum Synch Get Buffer Size (bytes):                             31680    
  Average Synch Get Buffer Size (bytes):                             31680    
  Number of Read Stream Requests:                                    3844     
  Minimum Read Stream Buffer Size (bytes):                           19       
  Maximum Read Stream Buffer Size (bytes):                           31744    
  Average Read Stream Buffer Size (bytes):                           14556    
  Minimum Accumulated Read Ahead Data Size (bytes):                  0        
  Maximum Accumulated Read Ahead Data Size (bytes):                  1172480  
  Average Accumulated Read Ahead Data Size (bytes):                  594385   
  Local Response Count:                                              20827    
  Average Local Response Time (usec):                                250895   
  Remote Response Count:                                             70486    
  Average Remote Response Time (usec):                               277036     
  Current 2000 Accelerated Sessions:                                 0        
  Current 2003 Accelerated Sessions:                                 1        
  Current 2007 Accelerated Sessions:                                 0        
  Secured Connections:                                               1               <-----Encrypted connections        
  Lower than 2000 Sessions:                                          0        
  Higher than 2007 Sessions:                                         0        

You can find the IP addresses of clients requesting encrypted MAPI connections in the syslog by searching for messages like the following:

2009 Jan 5 13:11:54 WAE512 mapi_ao: %WAAS-MAPIAO-3-132104: (929480) Encrypted connection. Client ip: 10.36.14.82

You can view the MAPI connection statistics by using the show statistics connection optimized mapi detail command as follows:

WAE674# show stat conn opt mapi detail
Connection Id:            1830
   Peer Id:                  00:14:5e:84:24:5f
   Connection Type:          EXTERNAL CLIENT
   Start Time:               Thu Jun 25 06:32:27 2009
   Source IP Address:        10.10.10.10
   Source Port Number:       3774
   Destination IP Address:   10.10.100.101
   Destination Port Number:  1146
   Application Name:         Email-and-Messaging                            <-----Should see Email-and-Messaging
   Classifier Name:          **Map Default**
   Map Name:                 uuida4f1db00-ca47-1067-b31f-00dd010662da       <-----Should see this UUID
   Directed Mode:            FALSE
   Preposition Flow:         FALSE
   Policy Details:
          Configured:        TCP_OPTIMIZE + DRE + LZ
             Derived:        TCP_OPTIMIZE + DRE + LZ
                Peer:        TCP_OPTIMIZE + DRE + LZ
          Negotiated:        TCP_OPTIMIZE + DRE + LZ
             Applied:        TCP_OPTIMIZE + DRE + LZ
   Accelerator Details:
               Configured:   MAPI                                           <-----Should see MAPI configured
                  Derived:   MAPI 
                  Applied:   MAPI                                           <-----Should see MAPI applied
                     Hist:   None
                                               Original            Optimized
                                   -------------------- --------------------
   Bytes Read:                                     4612                 1973
   Bytes Written:                                  4086                 2096
. . .


Local and remote response counts and average response times are shown in this output:

 . . . 
MAPI : 1830

   Time Statistics were Last Reset/Cleared:                           Thu Jun 25
 06:32:27 2009
   Total Bytes Read:                                                46123985
   Total Bytes Written:                                             40864046
Number of Synch Get Buffer Requests:                               0
Minimum Synch Get Buffer Size (bytes):                             0
Maximum Synch Get Buffer Size (bytes):                             0
Average Synch Get Buffer Size (bytes):                             0
Number of Read Stream Requests:                                    0
Minimum Read Stream Buffer Size (bytes):                           0
Maximum Read Stream Buffer Size (bytes):                           0
Average Read Stream Buffer Size (bytes):                           0
Minimum Accumulated Read Ahead Data Size (bytes):                  0
Maximum Accumulated Read Ahead Data Size (bytes):                  0
Average Accumulated Read Ahead Data Size (bytes):                  0
Local Response Count:                                              0                  <----------
Average Local Response Time (usec):                                0                  <----------
Remote Response Count:                                             19                 <----------
Average Remote Response Time (usec):                               89005              <----------
. . . 

MAPI AO Logging

The following log files are available for troubleshooting MAPI AO issues:

  • Transaction log files: /local1/logs/tfo/working.log (and /local1/logs/tfo/tfo_log_*.txt)
  • Debug log files: /local1/errorlog/mapiao-errorlog.current (and mapiao-errorlog.*)

For easier debugging, you should first set up an ACL to restrict packets to one host.

WAE674(config)# ip access-list extended 150 permit tcp host 10.10.10.10 any
WAE674(config)# ip access-list extended 150 permit tcp any host 10.10.10.10

To enable transaction logging, use the transaction-logs configuration command as follows:

wae(config)# transaction-logs flow enable
wae(config)# transaction-logs flow access-list 150

You can view the end of a transaction log file by using the type-tail command as follows:

wae# type-tail tfo_log_10.10.11.230_20090715_130000.txt
Wed Jul 15 19:12:35 2009 :2289 :10.10.10.10 :3740 :10.10.100.101 :1146 :OT :END :EXTERNAL CLIENT :(MAPI) :822 :634 :556 :706
Wed Jul 15 19:12:35 2009 :2289 :10.10.10.10 :3740 :10.10.100.101 :1146 :SODRE :END :730 :605 :556 :706 :0
Wed Jul 15 19:12:35 2009 :2290 :10.10.10.10 :3738 :10.10.100.101 :1146 :OT :END :EXTERNAL CLIENT :(MAPI) :4758 :15914 :6436 :2006
Wed Jul 15 19:12:35 2009 :2290 :10.10.10.10 :3738 :10.10.100.101 :1146 :SODRE :END :4550 :15854 :6436 :2006 :0
Wed Jul 15 19:12:35 2009 :2284 :10.10.10.10 :3739 :10.10.100.101 :1146 :OT :END :EXTERNAL CLIENT :(MAPI) :1334 :12826 :8981 :1031

To set up and enable debug logging of the MAPI AO, use the following commands.

NOTE: Debug logging is CPU intensive and can generate a large amount of output. Use it judiciously and sparingly in a production environment.

You can enable detailed logging to the disk as follows:

WAE674(config)# logging disk enable 
WAE674(config)# logging disk priority detail

You can enable debug logging for connections in the ACL as follows:

WAE674# debug connection access-list 150

The options for MAPI AO debugging are as follows:

WAE674# debug accelerator mapi ?
  all           enable all MAPI accelerator debugs
  Common-flow   enable MAPI Common flow debugs
  DCERPC-layer  enable MAPI DCERPC-layer flow debugs
  EMSMDB-layer  enable MAPI EMSMDB-layer flow debugs
  IO            enable MAPI IO debugs
  ROP-layer     enable MAPI ROP-layer debugs
  ROP-parser    enable MAPI ROP-parser debugs
  RPC-parser    enable MAPI RPC-parser debugs
  shell         enable MAPI shell debugs
  Transport     enable MAPI transport debugs
  Utilities     enable MAPI utilities debugs

You can enable debug logging for MAPI connections and then display the end of the debug error log as follows:

WAE674# debug accelerator mapi Common-flow
WAE674# type-tail errorlog/mapiao-errorlog.current follow

Rating: 5.0/5 (1 vote cast)

Personal tools