Emails Going to TXT Files on File Server

From DocWiki

Jump to: navigation, search


Contents

Scenario Setup

Administration Console

  1. Configure twitter@eim.lab as a Blocked Address

Agent PC

  1. Send mail from twitter@eim.lab to support@eim.lab
  2. Introduce an email to the retriever whose message-id is is already in the eGActiveDB

Problem Statement

Emails are being retrieved from the mail server, but they are not being routed to agents.

Spam Mails

A quick look at the Retriever logs indicates these are being marked as Spam.

2011-05-30 18:54:53.434 GMT-0400 <@> INFO <@> [6665:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Number of Messages:   1 <@> 
2011-05-30 18:54:53.574 GMT-0400 <@> WARN <@> [6665:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> checkAndLogSpamEmail<999> <@> SPAM_TYPE_ADDRESS   cimadmin@dslab.cisco.com <@> 
2011-05-30 18:54:53.762 GMT-0400 <@> INFO <@> [6665:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.platform.framework.event.eventhistory.EventHistory <@> insertEventData(2) <@> The Query Name is:   eventhistory::insert_spam_event_data <@> 
2011-05-30 18:54:53.777 GMT-0400 <@> INFO <@> [6665:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.platform.framework.event.eventhistory.EventHistory <@> insertEventData(2) <@> Event Logged without bulk insertion. <@> 
2011-05-30 18:54:53.777 GMT-0400 <@> WARN <@> [6665:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Spam Email, Message Id: 4DE4201A.7040501@dslab.cisco.com <@> 

While looking at the Retriever logs, you noticed a text file among the other process logs in the EIM/WIM Logs directory titled "RxSpamEmails.txt".

SpamLog.png


In this file, you see customer emails!

Alias: eim@DSlab.cisco.com, Server: 10.77.30.18
Reason: Rx: Dump spam email into file
Date: 2011-05-30 22:54:53 GMT
startemail-123987****a1b2c3d4****m1n2o3p4****egain
Return-Path: CIMAdmin@dslab.cisco.com
Received: from [10.77.30.8] ([10.77.30.8])
	by dslab.cisco.com
	; Mon, 30 May 2011 18:54:13 -0400
Message-ID: <4DE4201A.7040501@dslab.cisco.com>
Date: Mon, 30 May 2011 18:54:18 -0400
From: CIM Admin <CIMAdmin@dslab.cisco.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.4) Gecko/20100608 Thunderbird/3.1
MIME-Version: 1.0
To: eim@dslab.cisco.com
Subject: Test Email
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

  Testing123

endemail-123987****a1b2c3d4****m1n2o3p4****egain

Why would mail go to RxSpamMails.txt?

The only reason incoming emails would be written to this log is if all or part of the Sender's address matches a configured "Blocked Address." From the Administration Console, we can see that CIMAdmin@dslab.cisco.com is a blocked address. This is our root cause.

BlockedAddress.png


Partition Settings Group

The action EIM takes on spam emails is set in the Administration Console.

PSGspam.png


Malformed Emails

Another report of emails being retrieved from the mail server, but not being assigned to agents. This time, the Retriever Logs throw some scary SQL errors about duplicate keys.

2011-05-30 19:29:15.661 GMT-0400 <@> ERROR <@> [145490:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.platform.framework.dataaccess.DataAccess <@> executePreparedStoredProc() <@> I18N_EGPL_DATAACCESS-SQLEXCEPTION_WHILE_EXECUTING_QUERYViolation of UNIQUE KEY constraint 'EGML_EMAIL_MSG_ID_UQ'. Cannot insert duplicate key in object 'dbo.EGML_EMAIL'. <@> 
com.microsoft.sqlserver.jdbc.SQLServerException: Violation of UNIQUE KEY constraint 'EGML_EMAIL_MSG_ID_UQ'. Cannot insert duplicate key in object 'dbo.EGML_EMAIL'.
	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:197)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1493)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:390)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:340)
	at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
	at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:179)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:154)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:308)
	at com.egain.platform.framework.dataaccess.dbconnectionfactory.PreparedDBStatement.executeUpdate(PreparedDBStatement.java:40)
	at com.egain.platform.framework.dataaccess.dbconnectionfactory.CallableDBStatement.executeUpdate(CallableDBStatement.java:25)
	at com.egain.platform.framework.dataaccess.DataAccess.executeUpdateWithTimeoutCheck(DataAccess.java:6289)
	at com.egain.platform.framework.dataaccess.DataAccess.executeStoredProcForPreparedStmt(DataAccess.java:4151)
	at com.egain.platform.framework.dataaccess.DataAccess.executePreparedStoredProc(DataAccess.java:4054)
	at com.egain.mail.module.retriever.service.RxProcess.insertEmail(RxProcess.java:2433)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMailsForHost(RxProcess.java:2213)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMails(RxProcess.java:1386)
	at com.egain.mail.module.retriever.service.RxInstance.startInstance(RxInstance.java:114)
	at com.egain.platform.framework.dsm.common.ThreadPool$PoolThread.run(ThreadPool.java:112)
2011-05-30 19:29:15.661 GMT-0400 <@> ERROR <@> [145490:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.platform.framework.dataaccess.DataAccess <@> executePreparedStoredProc() <@> CallableStatement closed. DBConnection released due to some error. <@> 
2011-05-30 19:29:15.661 GMT-0400 <@> ERROR <@> [145490:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> insertEmail<999> <@> I18N_EGML_RETRIEVER-FAILED_TO_INSERT_EMAIL :Alias -> eim@DSlab.cisco.com <@> 
com.microsoft.sqlserver.jdbc.SQLServerException: Violation of UNIQUE KEY constraint 'EGML_EMAIL_MSG_ID_UQ'. Cannot insert duplicate key in object 'dbo.EGML_EMAIL'.
	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:197)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1493)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:390)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:340)
	at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
	at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:179)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:154)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:308)
	at com.egain.platform.framework.dataaccess.dbconnectionfactory.PreparedDBStatement.executeUpdate(PreparedDBStatement.java:40)
	at com.egain.platform.framework.dataaccess.dbconnectionfactory.CallableDBStatement.executeUpdate(CallableDBStatement.java:25)
	at com.egain.platform.framework.dataaccess.DataAccess.executeUpdateWithTimeoutCheck(DataAccess.java:6289)
	at com.egain.platform.framework.dataaccess.DataAccess.executeStoredProcForPreparedStmt(DataAccess.java:4151)
	at com.egain.platform.framework.dataaccess.DataAccess.executePreparedStoredProc(DataAccess.java:4054)
	at com.egain.mail.module.retriever.service.RxProcess.insertEmail(RxProcess.java:2433)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMailsForHost(RxProcess.java:2213)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMails(RxProcess.java:1386)
	at com.egain.mail.module.retriever.service.RxInstance.startInstance(RxInstance.java:114)
	at com.egain.platform.framework.dsm.common.ThreadPool$PoolThread.run(ThreadPool.java:112)
2011-05-30 19:29:15.661 GMT-0400 <@> ERROR <@> [145490:RxInstance id : 999] <@> ProcessId:5368 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> insertEmail<999> <@> I18N_EGML_RETRIEVER-DUPLICATE_EMAIL_RECEIVED: Message Id:eim@DSlab.cisco.com^4DE4250B.3060604@dslab.cisco.com <@> 

The last line is what you're really looking for: "DUPLICATE_EMAIL_RECEIVED". This means that that Message ID is already in the database under a different activity ID. You can validate this by running the following query on the eGActiveDB:

select * from egml_email_data where HEADER LIKE '%4DE4250B.3060604%'

The Message ID is in the header, and we can see that it was originally used in activity ID 1238.

EmailID.png


Information such as email size and charset can be obtained with the following query:

select * from egml_email where message_id like '%4DE4250B.3060604%'

EgmlEmail.png


Behavior Changed in 4.3(2) ES7

Note that the above error occurred in 4.3(2) ES5. In ES7, there is no longer a restriction on duplicate message-IDs. Since this class is using ES7, we will introduce the malformed email scenario in a different way. The retriever logs in our case will throw the below errors:

2011-07-18 15:42:22.208 GMT-0400 <@> ERROR <@> [49:RxInstance id : 999] <@> ProcessId:3676 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.Parse <@> parseTextPart <@> I18N_EGML_RETRIEVER-I_O_EXCEPTION  <@> 
java.io.UnsupportedEncodingException: ISO 8859-1
	at sun.io.Converters.getConverterClass(Converters.java:218)
	at sun.io.Converters.newConverter(Converters.java:251)
	at sun.io.ByteToCharConverter.getConverter(ByteToCharConverter.java:68)
	at java.lang.StringCoding.decode(StringCoding.java:224)
	at java.lang.String.<init>(String.java:405)
	at java.lang.String.<init>(String.java:433)
	at com.egain.mail.module.retriever.Parse.parseTextPart(Parse.java:1172)
	at com.egain.mail.module.retriever.Parse.dumpPart(Parse.java:760)
	at com.egain.mail.module.retriever.Parse.parseEmail(Parse.java:347)
	at com.egain.mail.module.retriever.service.RxProcess.parseEmail(RxProcess.java:4345)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMailsForHost(RxProcess.java:1956)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMails(RxProcess.java:1386)
	at com.egain.mail.module.retriever.service.RxInstance.startInstance(RxInstance.java:114)
	at com.egain.platform.framework.dsm.common.ThreadPool$PoolThread.run(ThreadPool.java:112)
2011-07-18 15:42:22.208 GMT-0400 <@> ERROR <@> [49:RxInstance id : 999] <@> ProcessId:3676 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.Parse <@> parseEmail <@> I18N_EGML_RETRIEVER-FAILED_EXCEPTION  <@> 
com.egain.mail.module.retriever.exception.FailedToParseEmailException: ISO 8859-1
	at com.egain.mail.module.retriever.Parse.parseTextPart(Parse.java:1286)
	at com.egain.mail.module.retriever.Parse.dumpPart(Parse.java:760)
	at com.egain.mail.module.retriever.Parse.parseEmail(Parse.java:347)
	at com.egain.mail.module.retriever.service.RxProcess.parseEmail(RxProcess.java:4345)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMailsForHost(RxProcess.java:1956)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMails(RxProcess.java:1386)
	at com.egain.mail.module.retriever.service.RxInstance.startInstance(RxInstance.java:114)
	at com.egain.platform.framework.dsm.common.ThreadPool$PoolThread.run(ThreadPool.java:112)
2011-07-18 15:42:22.224 GMT-0400 <@> ERROR <@> [49:RxInstance id : 999] <@> ProcessId:3676 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> I18N_EGML_RETRIEVER-FAILED_TO_PARSE_EMAIL : Alias :support@eim.lab <@> 
com.egain.mail.module.retriever.exception.FailedToParseEmailException: ISO 8859-1
	at com.egain.mail.module.retriever.Parse.parseEmail(Parse.java:376)
	at com.egain.mail.module.retriever.service.RxProcess.parseEmail(RxProcess.java:4345)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMailsForHost(RxProcess.java:1956)
	at com.egain.mail.module.retriever.service.RxProcess.retrieveMails(RxProcess.java:1386)
	at com.egain.mail.module.retriever.service.RxInstance.startInstance(RxInstance.java:114)
	at com.egain.platform.framework.dsm.common.ThreadPool$PoolThread.run(ThreadPool.java:112)
2011-07-18 15:42:22.271 GMT-0400 <@> WARN <@> [49:RxInstance id : 999] <@> ProcessId:3676 <@> PID:1 <@> UID:12 <@> HttpSessionId:  <@> com.egain.mail.module.retriever.service.RxProcess <@> retrieveMailsForHost<999> <@> Exception Email, Message Id: 4E01F2A1.3020900@eim.lab <@> 

Note the last line, "Exception Email"

Partition Settings Group

The action EIM takes on Exception emails is set in the Administration Console.

PSGx.png


Where did the email go?

The RxExcepEmails.txt is not in the logs directory and will not be easily noticed like the RxSpamEmails.txt was. In this case, the file is in C:\CIM\eService\storage\1\mail\Exception Emails\RxExcepEmails.txt. We can see our problematic charset email has been written to this file:

Alias: support@eim.lab, Server: eimAD
Reason: Unable to Parse email.
Date: 2011-07-18 19:42:22 GMT
startemail-123987****a1b2c3d4****m1n2o3p4****egain
Received: from [14.104.48.26] ([14.104.48.26]) 
	by eimAD.eim.lab
	; Wed, 22 Jun 2011 09:48:17 -0400
Message-ID: <4E01F2A1.3020900@eim.lab>
Date: Wed, 22 Jun 2011 09:48:17 -0400
From: Mister Customer <customer@eim.lab>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11
MIME-Version: 1.0
To: support@eim.lab
Subject: Customer Email
Content-Type: text/plain; charset=ISO 8859-1; format=flowed
Content-Transfer-Encoding: 7bit

TestMessage

endemail-123987****a1b2c3d4****m1n2o3p4****egain

What Can We Do With These Emails?

There is no known automated way to recover the emails that have been delivered to the exception log file. They will have to be manually re-submitted with one of the following options:

  1. Re-send the emails again to the EIM alias from another email client
  2. Have an agent create a new case, copy/paste the problematic email's content and customer email address, and send a reply.

Resolution

From the 4.3 Administration Guide:

  • Spam emails are emails that are sent from the addresses (email address or domain address) that are blocked in a department. When the Retriever Services picks up emails from the Mail Server, it checks the “from address” and if the address matches a blocked email address or domain address, it is marked as a spam email and that email is not retrieved. Use the settings described in this section to decide what you want to do with the spam emails.
  • Exception emails are the emails which the Retriever Service fails to parse or store in the database.

For more information on what constitutes an Exception Email, reference the 4.3 Troubleshooting Guide:

  • Messages are not according to RFC 822 message standards. For example:
    • Content type field is missing from the message header.
    • Message ID is missing.
    • There is no start boundary.
    • The mail header character set value is not recognized by Java Mail. For example, it contains characters like “iso 8859-1” whereas the correct format is “iso-8859-1”.
    • Duplicate Message ID

Rating: 5.0/5 (1 vote cast)

Personal tools