[Ironport C160] Need help on accessing the queue and/or Diagnostic logs

Unanswered Question
Aug 20th, 2010

Hi,

We have an issue on our 2 Ironports C160 cluster.

IR1 = Primary C160

IR2 = Secondary C160

We simulated a IR1 shutdown to send/receive mails from IR2. The 2 of them are in the same network in Public DMZ.

The test was ok, but when the IR1 came back, a few mails were still on IR2, ready to deliver.

Several delivernow didn't work. In the logs, all the infos are good (no filter problem, av OK).

Example for mail1688 :

Wed Aug 18 17:48:42 2010 Info: MID 1688 ready 3891 bytes from <[email protected]>
Wed Aug 18 17:48:42 2010 Info: MID 1688 matched all recipients for per-recipient policy DEFAULT in the inbound table
Wed Aug 18 17:48:43 2010 Info: MID 1688 interim verdict using engine: CASE spam negative
Wed Aug 18 17:48:43 2010 Info: MID 1688 using engine: CASE spam negative
Wed Aug 18 17:48:43 2010 Info: MID 1688 interim AV verdict using Sophos CLEAN
Wed Aug 18 17:48:43 2010 Info: MID 1688 antivirus negative
Wed Aug 18 17:48:43 2010 Info: MID 1688 queued for delivery


We did a diagnostic to trace a delivernow on the private interface of IR2, but cannot find the /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Tableau Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} tracking_diagnostics.tgz file (nor the /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Tableau Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} delivery_Log one)

How can i see the mails in the queue ?

Where can we find the diagnostic and delivery logs ?

Thanks fo any help on this.


Gohoungo Ruddy

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Christopher Smith Fri, 08/20/2010 - 07:01

Greetings,

When a message is listed as queued for delivery, it has been sent to the delivery queue on the appliance.  You can gain some understanding of what is in the delivery queue from the CLI.

The Command 'showrecipients' can give you details about the messages waiting to be delivered.

The Command 'tophosts' can be used to view the status of the top 20 recipient domains. This will display the number of active recipients, number of soft bounced and hard bounced messages as well the number of connections to a specific host. The data from tophosts will also give you an idea of the number of connections to a host and if the host is up or down as of the last delivery attempt.

The command 'hoststatus' will give you more specific details about a recipient domain, such as the last 5.xx error and what DNS information is being provided for that host.

Trouble shooting delivery issues will almost always require the use of the mail logs. I am including details of that process below.

You can search the logs to gather more information about the From,  To, Subject of the emails coming from this IP address that you're  interested in.

The name of the log is "mail_logs". You can see this in the [System Administration > Log Subscriptions > mail_logs].

There are several ways to access these logs.

1. Via the web browser.

- Go to [System Administration > Log Subscription].
- For the mail_logs, click on the ftp link to the right of mail_logs
-  If it gives you an error, go to "Network -> IP interface", select  the interface that you normally access to the Ironport on and turn on  the FTP/port 21 service.


2. From the command line,

- Using a ssh client like Putty and log onto the command line of the Ironport appliance via port 22/ssh.
- From the command line, type this to search for the IP

grep (press Enter)
The # of the "mail_logs"
Then enter the pattern to search, ie. 192.168.1.1 or [email protected]

For the next three questions, press enter and keep the defaults.

The search may take a bit of time to complete.

Once the output comes back, you can search either the ICID or the MID.

i.e

grep "ICID 123456" mail_logs


Once the output comes back, you can search for the MID

grep "MID 78901234" mail_logs

and so on.

You should be able to see the From, To, Subject from the MID
You should see the IP address and the HAT Sender Group from the ICID


3.  Another option is to ftp the mail_logs to a local machine(Desktop) and  use your own file/text editor to search for the IP addresses.

Here is a link to some Support Portal knowledge base articles that may be of use:

How can I determine the disposition of a message using the mail logs?
http://tinyurl.com/jb7z4

What is a Message ID (MID)?
http://tinyurl.com/ky3kf


How do I extract the SBRS score of a sender from the mail logs?
http://tinyurl.com/3xh3sl

In your case your probably specifically interested in this following,

How can I determine the disposition of a message using the mail logs?
http://tinyurl.com/jb7z4

Christopher C Smith

CSE

Cisco IronPort Customer Support

systemvsc Wed, 08/25/2010 - 04:48

Hi,

Thanks to you but we succeed in resolving this issue (Configuration Mode problem).

It seems that the Cluster Conf, after modfications, didn't have been duplicated on the 2nd Ironport (IR2).

Weoverrided the settings in the SMTP Routing Menu, apply fictive settings, then re-apply the Cluster mode with IR1 conf as Master conf.

All the mails have been received.

Regards.

Jason Meyer Wed, 08/25/2010 - 08:06

I'd let the Cisco/IronPort moderators validate this but sounds like you may have some cluster inconsistancies going on...  There is a command "clustercheck" that you can run to test for this and it will ask if you want to ignore or force the clusters to sync up...  you can 'cancel' out of it.   I stumbled across it and ran it on our two C660s that are clustered and it found an inconsistancy.  My eyes got kind of big and started doing research as to what I should do...  opened a support call on it and IronPort support had me force it...   Next check came up good and all is well...  I made a note to run 'clustercheck' occasionally on our environment and in 2 1/2 years I've had two..  both times I forced it and haven't had any issues..   Support will probably tell you or offer to look at the configuration file to see if you can find the inconsistancy..   PSPad works great to compare the configuration files of each machine..

Just my two cents worth,

Jason

Christopher Smith Wed, 08/25/2010 - 16:52

HI Jason, Selil,

Excellent follow up on this.  Cluster inconsistencies can cause a lot of headaches and confusion.  I have actually worked a few tickets where folks spent a lot of time trying to diagnose what appeared to be a network problem with their listener only to find out later it was actually a cluster inconsistency that was to blame.

In addition to clustercheck in the CLI  you can also verify settings for all levels, including cluster, group and machine , by using the clustershow command.

There are some scenarios than can be a bit confusing in the clustering world, especially if you get hung up in a loop about committing changes. If it seems like your going in circles or have reached a dead in give us a call or open a suppor ticket and we will be happy to help you get things sorted out.

Christopher C Smith

CSE
Cisco IronPort Customer Support 

Ken Stieers Tue, 10/11/2011 - 18:09

Bret,

Go to Network/Interfaces

Make sure that FTP is turned on for the interface that you use to manage the box.

Then go to System Administration>Log Subcriptions, the logs should show up as ftp links...

Also you shoudl be able to open up any FTP client, connect to the box, and get the files.

Deliver logs are in the mail_logs directory....

Actions

This Discussion