Customer Emails Not Being Retrieved

From DocWiki

Revision as of 21:20, 15 November 2011 by Ginod (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


Contents

Scenario Setup

Exchange Server

  1. Stop POP3 Service

System Console

  1. Stop Retriever process

Administration Console

  1. Change Partition Setting Maximum Email Size to 1 byte
  2. Change Partition Setting Maximum Body Size to 1 byte

Agent PC

  1. Send an email from customer@eim.lab to support@eim.lab

Problem Statement

Exchange administrators say emails are sitting on the server and not being retrieved by EIM.

Isolation

First, we need to isolate where the problem lies. Is there something wrong with the email server? The EIM Servers? Or network connectivity between any of them?

Are new activities being created in the eGActiveDB? Use the following query to check for these new emails. 

select top(10) egpl_routing_queue.queue_name, * from egpl_casemgmt_activity LEFT OUTER JOIN egpl_routing_queue on egpl_casemgmt_activity.queue_id = egpl_routing_queue.queue_id order by activity_id desc

In our scenario, no. The mail is never being inserted into the eGActiveDB. This means that we need to move on to our remaining possibilities:

  1. Retriever Process
  2. Network
  3. Email server

What process is responsible for "retrieving" mails?

Retriever process "retrieves" mails from the mail server. Confirm in the System Console that both process and instance are running.

Then check the retriever logs, where you will likely see some errors about the aliases you have configured: 

2011-05-17 17:24:23.357 GMT-0400 <@> ERROR <@> [73:RxInstance id : 999] <@> ProcessId:4664 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> connectToServer <@> I18N_EGML_RETRIEVER-FAILED_MESSAGING_EXCEPTION eim@DSlab.cisco.com <@> 
javax.mail.MessagingException: Connect failed;
  nested exception is:
	java.net.ConnectException: Connection refused: connect
	at com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:161)
	at javax.mail.Service.connect(Service.java:288)
	at javax.mail.Service.connect(Service.java:169)
	at com.egain.mail.module.retriever.service.RxProcess.connectToServer(RxProcess.java:2831)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMailsForHost(RxProcess.java:1552)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMails(RxProcess.java:1386)
	at com.egain.mail.module.retriever.service.RxInstance.startInstance(RxInstance.java:114)
	at com.egain.platform.framework.dsm.common.ThreadPool$PoolThread.run(ThreadPool.java:112)
Caused by: java.net.ConnectException: Connection refused: connect
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:367)
	at java.net.Socket.connect(Socket.java:524)
	at com.sun.mail.util.SocketFetcher.createSocket(SocketFetcher.java:231)
	at com.sun.mail.util.SocketFetcher.getSocket(SocketFetcher.java:189)
	at com.sun.mail.pop3.Protocol.<init>(Protocol.java:94)
	at com.sun.mail.pop3.POP3Store.getPort(POP3Store.java:214)
	at com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:157)
	... 7 more


EIM is not even getting the emails. Is it configured properly?

Each "alias" in EIM must be associated with a Retriever instance. This is how the Retriever knows what emails to pull when it connects to the mail server. Ensure that your alias is associated through the Administration Console. Note that any changes made here will require a restart of the Retriever instance.

AliasInstance.png


Network Connectivity with Mail Server

Ping

Let's first try the obvious: can you ping the mail server? Since the Retriever process runs on the Services Server, run the ping from there.

Ping.png

Telnet 113

Basic network connectivity is fine. What about port 113? Try to telnet to it from the Services Server.

Port113.png

Telnet 25

Something is wrong with port 113. Can you reach port 25?

Port25.png


Something is wrong with the POP3 server on port 113. (Note: This is assuming you've verified that the customer is using POP3 and not IMAP, which would typically be port 143).

Email Server Config

Check the status of your POP3 Service: is it running?

Pop3serviceNotRunning.png


This can also be seen from the Exchange System Manager and started from there.

Pop3stopped.png


Still no emails?

Your POP3 server is fixed and running, but the emails are still not entering the database and being routed to agents. What is the Retriever doing? Log in as PA and set up a System Monitor for the Retriever Process from the System Console.

  1. Name your monitor
    RxMon.png
  2. Select the rx-instance object
    RxMon2.png
  3. Select the attributes:
    1. Throughput
    2. Unable to Send
    3. State
    4. Last Run Time
    5. Emails Skipped
      RxMon3.png
  4. Save, then Run your monitor. You see the mail initially pegged under "Pending" before transitioning to "Emails Skipped"
    RxMon4.png

Why is the Retriever skipping the email?

The logs bring clarity to this condition. The Retriever sees the email but considers it a "Large Email." 

2011-07-13 16:35:45.514 GMT-0400 <@> INFO <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxInstance <@> startInstance <@> **** Rx Instance 999 Running **** <@> 
2011-07-13 16:35:45.514 GMT-0400 <@> INFO <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Processing alias: eim@DSlab.cisco.com <@> 
2011-07-13 16:35:45.530 GMT-0400 <@> INFO <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> POP3FlagDelete: true POP3FlagDeleteAll: true <@> 
2011-07-13 16:35:48.295 GMT-0400 <@> INFO <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@>  Folder to connect: inbox <@> 
2011-07-13 16:35:48.295 GMT-0400 <@> INFO <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Number of Messages:   1 <@> 
2011-07-13 16:35:49.077 GMT-0400 <@> WARN <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> isEmailLarge <@> Large Mail :602 Bytes <@> 
2011-07-13 16:35:49.077 GMT-0400 <@> WARN <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Large Email, Message Id: 4E1E014D.60704@dslab.cisco.com by eim@DSlab.cisco.com <@> 
2011-07-13 16:35:49.092 GMT-0400 <@> INFO <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxInstance <@> startInstance <@> **** Rx Instance 999 Waiting **** <@>

Mail Criteria - Size

One restriction placed on emails coming in to the system is size. Administrators can configure the maximum email and body size of emails being processed. Note: The Retriever Instance must be restarted for these changes to take effect.

  • "Maximum email size for retriever (bytes)" WILL prevent an email from arriving. Emails exceeding this limit will not be retrieved from the mail server.
    • For example, if a 5MB email is on the server, and the Maximum Email Size is set to 2MB, then the Retriever will not pull the email off the server.
      MaxEmailSize1B.png
      A notification email will be sent to EIM administrators informing them of the large email.
      TestEmailMaxEmailSize.png
  • In this scenario, we can see that our "Maximum email size for retriever" is incorrectly set to 1 byte. Log in as PA to reset this to a more reasonable value such as the default of 2097152 bytes, and restart the Retriever Instance.
  • Log in as an agent and go ready for other channels to receive your email.

Email Arrives at Agent without a body!

Emails are finally being pulled off the mail server, but the body appears to agents as "Email body was too large. It is saved as an attachment." Agents must click the "View Attachments" button to open the email in a new window.

TestEmailMaxBodySizeArrow.png


The Retriever logs show a different warning message, this time stating that the content is too large: 

2011-07-13 16:43:12.353 GMT-0400 <@> WARN <@> [75:RxInstance id : 999] <@> ProcessId:7108 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> replaceContentWithMessageNoteIfLarge <@> Message content too large   78 <@>

We have another configuration item to check.

  • "Maximum body size for retriever (bytes)" will NOT prevent an email from arriving. It will simply take a large-body email and convert it to an attachment.
    • For example, if a 100KB email comes in, and the Maximum Body Size is set to 512 bytes, then EIM will convert the email (text, images, etc) to attachments.
      MaxBodySize.png
  • In this scenario, we can see that our "Maximum body size for retriever" is incorrectly set to 1 byte. Log in as PA to increase this (default 102400 bytes) to allow email bodies to show in the reply pane, and restart the Retriever Instance.


How High is Too High?

Note that setting the body and email sizes too high may cause issues with the Retriever process. For example, assume the Max Email and Max Body sizes were both set to 300097152 bytes.

An extremely large text email could cause the Retriever's java process to crash. In the below example, we can see that a 16.3MB text email has singlehandedly brought down the Retriever process.

LargeEmail.png 


2011-06-05 14:51:17.077 GMT-0400 <@> INFO <@> [66:RxInstance id : 999] <@> ProcessId:2304 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Processing alias: eim@DSlab.cisco.com <@> 
2011-06-05 14:51:17.077 GMT-0400 <@> INFO <@> [66:RxInstance id : 999] <@> ProcessId:2304 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> POP3FlagDelete: true POP3FlagDeleteAll: true <@> 
2011-06-05 14:51:17.155 GMT-0400 <@> INFO <@> [66:RxInstance id : 999] <@> ProcessId:2304 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@>  Folder to connect: inbox <@> 
2011-06-05 14:51:17.155 GMT-0400 <@> INFO <@> [66:RxInstance id : 999] <@> ProcessId:2304 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Number of Messages:   1 <@> 
2011-06-05 14:51:17.921 GMT-0400 <@> WARN <@> [66:RxInstance id : 999] <@> ProcessId:2304 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.Parse <@> dumpEnvelope <@> Exception While Setting Custom Header Field or No Custom Header Found <@> 
2011-06-05 14:51:24.640 GMT-0400 <@> ERROR <@> [66:RxInstance id : 999] <@> ProcessId:2304 <@> PID:-1 <@> UID:-1 <@> HttpSessionId:  <@> com.egain.platform.framework.dsm.common <@> run() <@> Instance Failed. Exiting.. == >999 <@> 
java.lang.OutOfMemoryError: Java heap space
	at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:99)
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:393)
	at java.lang.StringBuilder.append(StringBuilder.java:120)
	at com.egain.mail.module.retriever.Parse.parseTextPart(Parse.java:1196)
	at com.egain.mail.module.retriever.Parse.dumpPart(Parse.java:760)
	at com.egain.mail.module.retriever.Parse.parseEmail(Parse.java:347)
	at com.egain.mail.module.retriever.service.RxProcess.parseEmail(RxProcess.java:4345)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMailsForHost(RxProcess.java:1956)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMails(RxProcess.java:1386)
	at com.egain.mail.module.retriever.service.RxInstance.startInstance(RxInstance.java:114)
	at com.egain.platform.framework.dsm.common.ThreadPool$PoolThread.run(ThreadPool.java:112)

Note that the retriever process will automatically restart, but will not attempt to pull that mail again by default. 

2011-06-05 15:06:24.155 GMT-0400 <@> INFO <@> [47:RxInstance id : 999] <@> ProcessId:2364 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Processing alias: eim@DSlab.cisco.com <@> 
2011-06-05 15:06:24.155 GMT-0400 <@> INFO <@> [47:RxInstance id : 999] <@> ProcessId:2364 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> POP3FlagDelete: true POP3FlagDeleteAll: true <@> 
2011-06-05 15:06:24.218 GMT-0400 <@> INFO <@> [47:RxInstance id : 999] <@> ProcessId:2364 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@>  Folder to connect: inbox <@> 
2011-06-05 15:06:24.218 GMT-0400 <@> INFO <@> [47:RxInstance id : 999] <@> ProcessId:2364 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Number of Messages:   1 <@> 
2011-06-05 15:06:24.608 GMT-0400 <@> INFO <@> [47:RxInstance id : 999] <@> ProcessId:2364 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxInstance <@> startInstance <@> **** Rx Instance 999 Waiting **** <@>

Also note that EIM's action on large emails can be changed to delete instead of skip from the Administration Console:

PSGlarge.png

Resolution

In this scenario we focused on the troubleshooting required when emails are not being pulled into EIM.

  • Issues with POP3 server (note: the same logic could be applied to IMAP servers, with different port numbers)
  • Maximum Email Size configured on EIM preventing message retrieval
  • Maximum Body Size configured on EIM preventing display of messages in Agent Console's reply pane

Rating: 0.0/5 (0 votes cast)

Retrieved from "http://docwiki.cisco.com/wiki/Customer_Emails_Not_Being_Retrieved"
Personal tools