Retriever issues

From DocWiki

Revision as of 09:42, 2 May 2014 by Mgundapp (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Retriever issue – E-Mails not getting retrieved

Primary steps

  • Retriever (Rx) process and instance(s) running? (check from Unified EIM / Unified WIM System Console)
  • Check validity of alias details. Try to telnet to POP3 server using the following command (110 is the POP3 port number) and input alias credentials at login prompt: telnet <server name> 110

Monitoring - Instance Level Information

  • Last Run Time less than Start Time in System Monitor? (see Unified EIM / Unified WIM System Console User Guide for steps to create a monitor).
  • Retriever instance(s) may have hung. Check <Unified EIM / Unified WIM Install Dir>/logs/rx-process.log and <Unified EIM / Unified WIM Install Dir>/logs/eg_log_<server_name>_rx-process.log for JAVA exceptions and time-out messages.

Last Run Time much lesser than Current Time

  • Rx instance hung due to problematic E-Mail or Rx has connection problems.
  • Rx instance crashed due to some error viz. OutOfMemoryError. Check <Unified EIM / Unified WIM Install Dir>/logs/eg_log_<server_name>_rx-process.log and <Unified EIM / Unified WIM Install Dir>/logs/eg_log_<server_name>_rx-process.log for string ‘Instance failed’.

Monitoring - Alias Level Information

  • Check <Unified EIM / Unified WIM Install Dir>/logs/rx-process.log and <Unified EIM / Unified WIM Install Dir>/logs/ eg_log_<server_name>_rx-process.log for alias related errors such as login failures, alias being inactive/disabled, etc. Search for occurrences of the string “RETRIEVER”

Monitoring - Alias Level info

Alias state

  • Inactive
  • Login Failed
    • Authentication failure
  • Disabled
    • Mail.Retriever.RetryCount preference
    • 'Repeated Authentication Failure. Alias disabled‘ error in logs
    • Modify alias and save. Process restart is not required
  • Connection Failed
    • Wrong server name or port mismatch
    • Mail server may be down
    • POP3 or IMAP service may not be running
    • Connection between services server and mail server may be a problem – check using telnet client

E-Mails skipped

  • Spam E-Mails
  • Parsing/Insertion E-Mail errors – Exception E-Mails (genuine E-Mails, not to be confused with Undeliverable E-Mails/Delivery Exceptions that get routed to Exception queue).

Important Logs

  • <Unified EIM / Unified WIM Install Dir>/logs/batch-exec-rx-process.txt
  • <Unified EIM / Unified WIM Install Dir>/logs/eg_log_<server_name>_rx-process.log

RxException E-Mails

  • Parsing failures
    • messages not adhering to RFC 822 message standards. E.g., Content type field is missing in the message header
    • missing message id
    • no start boundary
    • The mail header character set value is not recognizable by Java Mail, E.g., iso-8859-1 Vs. iso 8859-1.
  • Insertion failures due to duplicate message id
    • Compare the timestamp and server names in the delivery path (in the initial few lines of the header) with that of the mail in the DB. If they are exactly the same, this signifies that there is some problem with deletion of the E-Mail from the server. Deletion failure happens when there is a network failure while closing the mailbox folder.
    • If they are different, it is a problem at the mail server level, and can be ignored.

Other Issues:

Java Mail Debug Mode

  • Set Rx.Debug = 1 in <Unified EIM / Unified WIM Install Dir>/config/ file. Needs restart of Rx process.
  • Logs appear in rx-process.log file

E-Mails getting skipped because of large E-Mail size

  • E-Mail size on mail server + buffer size < max E-Mail size (Default = 20 MB)
  • Notification on skipped E-Mails can be configured to go to an intended recipient (see screenshot showing partition settings for Retriever), and the setting can also be configured to delete or retain (after marking the E-Mail as skipped) such E-Mails on POP3.

Rich text formatted E-Mails are displayed in plain text

  • Microsoft Exchange Server provides a setting to convert rich text to html

Attachments not retrieved

  • Check department level preference to see if all attachments are blocked. Also, check the list of blocked attachments.
  • Action on blocked attachments = quarantine, delete, etc.

Best Practices:

Rx process – sizing

  • Mail size Vs JVM heap size
    • JVM Max size: 128 MB -> E-Mails retrieved: 5MB x 10/cycle
    • JVM Max size: 256 MB -> E-Mails retrieved: 10MB x 12/cycle
    • JVM Max size: 512 MB -> E-Mails retrieved: 15MB x 10+/cycle
    • JVM Max size: 512 MB -> E-Mails retrieved: 25MB x 6/cycle
    • Changing Rx process JVM Heap Size in <Unified EIM / Unified WIM Install Dir>/config/egpl_dsm.xml
  • One complete run of the Retriever before it enters into a “wait” state before the next run is referred to as a cycle.
  • Avoid configuring many instances per process, to prevent memory errors.
  • Limit each Retriever process to consist of 4 instances. Configure a new Retriever process when more than 4 instances are required

Distribute aliases across instances based on incoming E-Mail load

  • Multiple threads of the Retriever process (these are referred to as instances) can be created such that all retrieve E-Mails from different aliases on the POP3 concurrently.
  • Avoids slowness in processing E-Mails, reduces cycle time, and increases throughput
  • Prevents memory contentions with having just a single instance.
  • High volume aliases would be dedicated to a given instance.

Useful Informations:

Notification Scenarios

  • Mails get inserted into <Unified EIM / Unified WIM Install Dir>/logs/RxExcepMails.txt
  • Large E-Mails get skipped
  • Blocked attachments get deleted
  • When the Mail Server IP/Host Name/Port is incorrect, or Retriever fails to connect to the POP3/Exchange server due to a network failure.
  • When the username or password is incorrect, and Retriever fails the authentication with Mail Server.
  • If the Retriever is configured to connect to a mailbox folder other than the default, and that folder does not exist on the Mail Server.
  • If Retriever fails to get the number of E-Mails in the mailbox.
  • If the number of pending E-Mails in the mail box exceeds the defined maximum number of pending E-Mails for all or specific aliases.

Restart scenarios

  • Retriever crashes due to error, e.g., lack of memory
  • Retriever hangs due to DB deadlocks, or after waiting indefinitely for a response from the Mail Server.

Important settings

  • Mail.Retriever.SocketTimeOut – Connection or socket timeout exceptions
  • Mail.Retriever.NumBatches - Number of E-Mails to retrieve
  • Mail.Retriever.LargeE-MailSizeBuffer
  • Max Retriever.NotificationE-Mails.FromAddress
  • Max Retriever.NotificationE-Mails.ToAddress
  • Max Retriever.ServiceFailoverDelay

Important properties (

  • Rx.IOTimeOut – Related to read timeout interval in milliseconds after a connection has been established (Default = 300000)
  • Rx.NumE-MailPending – Notification gets sent after this number of E-Mails is pending retrieval (Default = 500).

Telnet commands for POP3

  • telnet <hostname> 110 – to connect
  • user <user_name>
  • pass <password>
  • list - will list all messages in inbox. Each row will have <serial no> and size in bytes
  • retr <serial no> - to retrieve the mail
  • top <serial no> n – n number of lines of content retrieved after the header. If n=0, only header is retrieved
  • dele <serial no> - flags E-Mail for deletion
  • quit – closes the inbox and deletes the flagged E-Mails
  • Once logged in, changes to the POP3 mailbox are not reflected till a re-login

Rating: 0.0/5 (0 votes cast)

Personal tools