Outgoing Spam on ISP Network

Unanswered Question
Apr 14th, 2008
User Badges:

Hello,

I manage an ISP network with several C-Series appliances and as you might be aware, there are much infected hosts on ISP networks and therefore they send out spam.

This results in getting the outgoing IronPort IPs blacklisted.

I'm having some difficult blocking these spam/virus because if i turn antispam ON for outgoing mails, about 2/3 of outgoing mails are false positives. I don't know why IPAS classify these mails as spam.
I did put some rate-limiting,but this doesn't always help.

Does anyone of you have any recommendations on how i can minimize this problem?

Thanks,
Vinesh

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Doc_ironport Mon, 04/14/2008 - 19:53
User Badges:


I'm having some difficult blocking these spam/virus because if i turn antispam ON for outgoing mails, about 2/3 of outgoing mails are false positives.


That certainly doesn't sound right! We have many ISPs, large and small, who use IPAS to filter outgoing mail without any issues.

I would suggest making sure that your clients IP address are hitting a RELAY profile so that their SBRS score isn't being used as a part of the IPAS calculation, but even that shouldn't cause the rates to be as high as 2/3 false positives.

Probably worth a call to support to get them to look at the details in your configuration - something is obviously wrong there...
Vinesh_ironport Tue, 04/15/2008 - 04:15
User Badges:

Hello,

We do have these mails going through a relayed policy and we turned Senderbase control to OFF.
We even configured a relaylist for corporate and another for all others(where we applied more strict rules.
I'll definitely contact support so that they look into it.

Thanks,
Vinesh

staylor_ironport Wed, 04/16/2008 - 16:33
User Badges:

My question is that most infected machines (Botnets) won't use the ISP relay, they will simply try to send out direct to the internet on port 25, only valid mail clients would know what the IP address is of the IronPort's for relaying mail out.

Vinesh_ironport Fri, 04/18/2008 - 05:49
User Badges:

Hi,

Perhaps getting the S-Series on the network might help in sustaining the infected hosts/spyware,etc.

Could this help?What do you think?

Vinesh

staylor_ironport Fri, 04/18/2008 - 07:43
User Badges:

Absolutely, the L4TM Spyware audit will tell you who's infected and who's clean. Contact your local friendly Sales Representative for more information.

Donald Nash Fri, 04/18/2008 - 23:26
User Badges:

Monkeymadness is right: bots don't usually use a relay host, they send directly. If your IronPorts are getting blacklisted, then it may be because your whole network is getting blacklisted.

Have you checked your mail_logs to verify that the spam is actually traversing your IronPorts? If it's not, then identifying and individually blocking the infected systems may be feasible, depending on how many there are. If there are too many then your only recourse may be putting in a block on outbound port 25 connections. But that will cause problems for anyone on your network who runs a legitimate mail server, so it isn't to be done lightly.

Doc_ironport Sat, 04/19/2008 - 00:38
User Badges:

Most ISPs are finally coming to the realization that the only way to stop spam getting out of their network is to block port 25 outgoing.

The vast majority of spam isn't going to be routed via an ISPs mail server (where it's much easier to detect/stop), but instead is going to try and go directly to the remote mail server.This makes detection difficult at best.

Like it or not, blocking outgoing port 25 to all hosts other than the ISPs mail servers is the only real choice. Giving customers an "opt-out" mechanism to this is a good idea, but to be effective it needs to default to blocked, and allow customers to unblock it if they want to do something like run their own mail servers - and for the most part those that unblock it are less likely to get infected with malware.

Is customers of an ISP want to send using another ISP/corporates mail server, that's what port 587 port is for - authenticated mail injection.

This is a big call for most ISPs to make, but it's the right one, and the only one that fixes this problem. I've spoken to many ISPs who have done this or are considering doing it , and in the end those that do it put up with some small pain for a while - but in the long run it saves them significant trouble!

Vinesh_ironport Sat, 04/19/2008 - 05:32
User Badges:

Indeed, the spam goes through the IronPorts and just yesterday , i deleted about 40,000 recipients which were queued for the @subdomain.hinet.com and some other domains.

Good idea, but concerning the opt-out option you are suggesting, the ISP needs to exclude these senders on their firewall i guess and it could be hard managing?
Also, for your users who have their mails hosted outside(mostly in US), how do they connect to their hosting mail servers. I would guess that they wouldn't like the idea of changing this and also not sure whether all these hosting companies accept other ports apart from 25.

Can you pls elaborate on the port 587 implementation? I've configured relaylist on the same incoming listeners. So, do i need to create an outbound listener and use port 587 on that in order to receive mails from subscribers?


I also have 2 other ISPs as client and this problem is also applicable to them. One of them is already using SMTP authentication and it's eliminating some of the spam problem. But the biggest ISP has about 50,000 subscribers i would guess(both corporate and home users) and implementing the port blocking approach might be quite hard. But i'll do share your recommendations with the ISP guys.

Thanks,
Vinesh

Donald Nash Sat, 04/19/2008 - 15:36
User Badges:


Most ISPs are finally coming to the realization that the only way to stop spam getting out of their network is to block port 25 outgoing.

But it is a difficult transition to make when you've got customers who are already making legitimate use of outbound port 25 connections. Yes, you can make an opt-out available, but you've got a customer communication problem letting them all know what is going to happen and what they need to do to prepare for it. That's why I said it isn't to be done lightly.

Indeed, the spam goes through the IronPorts

That is very strange, but in a way it works to your advantage: your own logs will tell you where the spam is coming from. But beyond that, you really need to open a support case to figure out why your IronPorts are not catching it.

Good idea, but concerning the opt-out option you are suggesting, the ISP needs to exclude these senders on their firewall i guess and it could be hard managing?

It's not trivial, but it's not impossible either. Many major ISPs here in the US do it. You just need some good management tools for keeping up with the list of exceptions to put into your firewall.

Can you pls elaborate on the port 587 implementation?

Port 587 is the SMTP submission port. It is the port where mail client programs (Thunderbird and the like), connect in order to submit their outbound mail. Submission SMTP is just like regular SMTP that runs on port 25, but with slightly different semantics. The most notable is that submission SMTP usually requires authentication for any mail transaction.
Vinesh_ironport Sat, 04/19/2008 - 18:01
User Badges:

Don,

I did contact Support and sent them some samples of the false positives.
They found out that the mails coming from teh subscribers are from blacklisted IPs(they also have very bad SBRS) and IPAS also evaluate this info about the blacklisted IP in their decision.
So, eventually, IPAS will surely see the mails as spam.
But they recommended an approach: Putting an incoming relay to be used as outgoing server by the subscribers and have the relay forward the mails to IronPort to be send to Internet. This way , IronPort will see the mails coming from the incoming relay and not from the blacklisted IPs.
What do you think?

As for the SMTP Auth and 587 port, the ISP told me that it will be hard to make all their users make this change. There would be some hard work involved !!

Thanks,
Vinesh

seveneyes_ironport Sun, 04/20/2008 - 01:06
User Badges:

I also look after 2 C650 appliances that service ISP customers in our dynamic address space. We had been using Brightmail until recently. When we attempted to implement IPAS we found as you did that a large percentage of the email was marked as spam. We found that we had to disable SenderBase IP Profiling on the listener. When this was done the false positive rate fell within what we were seeing for Brightmail. It appears with SenderBase IP Profiling active IPAS uses this info and marks many messages as spam due to the low SBRS of most IP addresses within the dynamic address ranges.

To prevent blacklisting of the Ironport IP addresses we have rate limiting applied to dynamic IP addresses that send via the ISP relay. This is done by adding all our dynamic space to the hat and using a dynamic policy. The rate limit is done setting the mail flow policy flow control to
Use SenderBase for Flow Control:off
Group by Similarity of IP Addresses:32

We currently block port 25 outbound for our dynamic space, but there are bots that search for and discover the relay. We deal with these customers by specifically blocking them from the relay until they clean their machine.

Vinesh_ironport Sun, 04/20/2008 - 04:31
User Badges:

Hello again,

Well interesting.
IN fact, i did create 2 relaylists. 1 with corporate clients IP range and another with home users(ADSL, Dial-up,etc having dynamic IPs). I've enabled strict rate-limiting on the second relaylist and i should say that we do throttle these bad senders. But eventually, some do get through because there are lots of infected hosts.

We already have this setting in place:
Setting the mail flow policy flow control to
Use SenderBase for Flow Control:off
Group by Similarity of IP Addresses:32


We have a public listener with the relaylist included. No dedicated private listeners for outbound mails. If i disable teh Senderbase IP Profiling, this will apply on incoming mails as well and could be a problem, isn't it?Perhaps IPAS will not evaluate these dynamic IPs SBRS for incoming.

Just to give you an idea, i've created an outgoing mail policy and included only one of the domains which the ISP hosts and on which i see lots of spoofed email addresses sending spam. From saturday 7am to sunday 7am, we got 10.2k spam and 580 virus on outgoing from one domain only.


Thanks,
Vinesh

seveneyes_ironport Sun, 04/20/2008 - 06:42
User Badges:

Our C650's only have a single public listener and they only deal with email being relayed from our DSL customers. The Ironport itself does not relay the email but passes it to a second tier of MTAs which do the relaying using smtproutes. Also important to mention is that the Ironport server has a firewall in front that restricts port 25 access to IPs in our network only. Roaming customers are expected to use secure SMTP with authentication which is on a different IP/listener on the same Ironport servers.

How many customers do you have? We don't see anything like these volumes. Almost no virus traffic at all. This may be because we closed port 25 in/out in our dynamic space (except for the SMTP relay) and also some well know ports that are commonly used to infect clients. There are still infected clients that try to send through our SMTP relay, but a lot of the traffic is caught by IPAS and rate limiting holds down the level if it does happen.

The vast majority of spam that hits our SMTP relay is from business customers that have their own email servers and are using our relay as a smart host. Almost all these messages are NDR messages containing spam from MS SMTP servers because recipient checking is not done during conversation.

Donald Nash Mon, 04/21/2008 - 16:09
User Badges:


But they recommended an approach: Putting an incoming relay to be used as outgoing server by the subscribers and have the relay forward the mails to IronPort to be send to Internet. This way , IronPort will see the mails coming from the incoming relay and not from the blacklisted IPs.
What do you think?

I think that would probably solve the false positive problem, but I have to wonder why Doc's original suggestion wouldn't do the same thing. Putting your clients' IP addresses into a separate sender group with SenderBase turned off should accomplish the same thing.

As for the SMTP Auth and 587 port, the ISP told me that it will be hard to make all their users make this change. There would be some hard work involved !!

Indeed there could. That's why I've been saying all along that resorting to port 25 blocking should not be done lightly. Everyone has to change what they're doing. But just because it is hard work doesn't mean it shouldn't be done. It just needs to be done with very careful planning and communication with your clients.
meyd45_ironport Fri, 04/25/2008 - 11:04
User Badges:

As seveneyes says, you need to turn off fetching SBRS on the listener.

In my experience SBRS less than -3 could cause a perfectly legitimate message to be marked spam +ve.

There is a bug/FR to be able to tell IPAS not to use SBRS even if it is available.

Vinesh_ironport Fri, 04/25/2008 - 14:32
User Badges:

Thanks to all,

I should be getting a C350 for this client by next week and i shall configure an outbound listener and disable the senderbase profiling and see how it works.

I'll keep you updated on the outcome.

Rgds,
Vinesh

mychrislo_ironport Sun, 05/04/2008 - 13:03
User Badges:

recipient control

Did you check this feature. This limits recipient per hour by your customer end's IP.

This isn't exactly a perfect solution, but I think this is better than no control.

We do block our subscriber's port25 and it did not solve the problem.
The real thing should be SMTP AUTH.

Btw, currently we fall back to sendmail for rate/connection control.

I bought up this thread a while ago, but seems ironport did not really consider to put similar measure into the AsynOS.

https://www.ironportnation.com/forums/viewtopic.php?t=375&start=0&postda...

RichardLindahl_... Tue, 05/06/2008 - 15:41
User Badges:

Well, I'd hate to be a partypooper but I wouldnt say smtp auth solves the entire problem.

We do port25 blocking for our customers so they have to use the a pair of
ironport x1000 configured with rate limiting and spamfiltering as outbound servers,
and eventhough this makes it easier to find the bad customers it doesnt stop misconfigured
mailservers that the customers have.

Say that a customer has a mailserver (misconfigured as an open relay)at home and
it relays through us using smtp auth. Without rate limiting we would get flooded with spam
from the customer in question. We usually find a few of those every month...

The last few weeks I've also been pestered with mail coming from networks outside
of our own customer networks using hacked mailaccounts using smtp auth to send spam
through our machines. So it would seem that the spammers have moved on as well...

But I'd have to agree that smtp auth slows down quite a few zombie machines :)

bfayne_ironport Thu, 05/15/2008 - 16:18
User Badges:


Hello,

We do have these mails going through a relayed policy and we turned Senderbase control to OFF.
We even configured a relaylist for corporate and another for all others(where we applied more strict rules.
I'll definitely contact support so that they look into it.




Make sure that you disabled Senderbase in the listener. Just turning it off in a Sender Group is not enough.

In the GUI, select Network/Listeners/Advanced and look for this option "Use SenderBase IP Profiling".

If you uncheck that box, Senderbase will not even be queried. Otherwise the ESA will always query SBRS and give that info to IPAS.

That solved a major false positive issue for me.
Vinesh_ironport Thu, 05/15/2008 - 17:03
User Badges:

Hello,

I did get a C350 and routed only outgoing mails for the entire ISP network on it with the SenderBase IP Profiling turned OFF on the listener and i noticed that it effectively reduced the number of false positives.

I should note that we do have some false positives and i'm still trying to find the correct IPAS threshold. But i have to admit that it's much better than previously and we are catching quite a lot of outgoing spam per day(coming from ADSL IPs and even corporate networks !! )

Thanks,
Vinesh

Actions

This Discussion