Very interesting. I didn't realize that it was that long ago that this work was done. I agree that it needs to be updated.

I will definitely look at doing this. My architecture could make this a bit easier.

Thanks

Side Notes: this was a few years back, I believe we were running AyncOS 3.8 at the time with limited reporting.  Also we counted invalid recipients in the 2.8 number.  I don't know of anyone outside of IronPort who has done the kind of experiment lately?  At the time our reasoning for doing this was to come up with what we felt was an approximate multiplying number for estimating inbound load to Exchange without Senderbase blocking.

In summary, I agree a multiplier of 3x does seem high at first, but realize that is 3 recipients estimated recipients per blocked inbound connection, not per blocked inbound message.  Meaning, on each blocked connection you don't know how many messages would be sent and/or how many recipients per message.  I would expect that this could be worse today because of the fact, giving an open door spammers will flood the target.

If you want to get your own accurate multiplier for ROI on blocking using SBRS, go for it.  Just except all the email with a high number of messages per connection and number of recipients per message, and do some log analysis.  Let us know what you get and feel free to modify the multiplier on your system.

Erich

 - edited for spelling errors.

Sorry if I confused things with my details above. The Dell work was impendent data; I just referenced this as an external finding. IronPort came up with the 3x multiplier based on IronPort's own findings.

Pat,

My 2 cents, and everyone has their own opinion, but I personally like the "multiplier" value to be 1, it's just my individual preference.

There is actually a history to this setting in addition to what Erich has added to the conversation. For those IronPort customers that remember code prior to Async OS 4.5.5 (pre-stacked bar graph reporting) we didn't really count stopped by reputation filtering. This was bad because other solutions (let's call it Solution A) that accepted every e-mail message and performed Anti-Spam processing would report (for example) catching 80% spam while a pre-4.5.5 IronPort system would report catching 30% spam (because SenderBase dropped a bunch before AS processing)...the problem was that when security and e-mail administrators took this information to the boss the typical response from the boss was that "Solution A" was catching more spam which forced admins perform research and report generation in order to correctly identify how much spam both Senderbase and IronPort Anti-Spam stopped. So IronPort performed research and the numbers ranged from 1.8 to 2.8 recipients per TCP connection and so the multiplier of 3 was created in order to demonstrate the value and actions of Senderbase Reputation Filtering.

So that's the history of why the value was created but personally I like to keep my value set to "1" just so I can get a more level'ized view of the statistics.

Sincerely,

Jay Bivens
IronPort Systems

@Jay: Thank you for this background-information!
@Erich:

Just except all the email with a high number of messages per connection and number of recipients per message, and do some log analysis

Sorry for asking, but HOW can I do that? I don't have that much experience with the C100 yet... :oops:

Example and command output below:

ip1.bivens.us> reportingconfig


Choose the operation you want to perform:
- MAILSETUP - Configure reporting for the ESA.
[]> mailsetup

SenderBase timeout used by the web interface: 5 seconds
Sender Reputation Multiplier: 3
The current level of reporting data recording is: unlimited
No custom second level domains are defined.


Choose the operation you want to perform:
- SENDERBASE - Configure SenderBase timeout for the web interface.
- MULTIPLIER - Configure Sender Reputation Multiplier.
- COUNTERS - Limit counters recorded by the reporting system.
- THROTTLING - Limit unique hosts tracked for rejected connection reporting.
- TLD - Add customer specific domains for reporting rollup.
[]> multiplier

Enter Sender Reputation Multiplier:
[3]> 1

SenderBase timeout used by the web interface: 5 seconds
Sender Reputation Multiplier: 1
The current level of reporting data recording is: unlimited
No custom second level domains are defined.


Choose the operation you want to perform:
- SENDERBASE - Configure SenderBase timeout for the web interface.
- MULTIPLIER - Configure Sender Reputation Multiplier.
- COUNTERS - Limit counters recorded by the reporting system.
- THROTTLING - Limit unique hosts tracked for rejected connection reporting.
- TLD - Add customer specific domains for reporting rollup.
[]>


Choose the operation you want to perform:
- MAILSETUP - Configure reporting for the ESA.
[]>

ip1.bivens.us> commit "Changed Reputation Multiplier"

Thank you again for the fast and very detailed answer! I will try that in the office next monday.

We keep 30 days worth of mail.text logs, and once a month we run a perl script on them. The script finds all messages with positive spam scores, eliminates those which arrived on SMTP connections that were subjected to rate limiting (since this would skew the results), and then calculates the average recipients/connection, messages/connection, and recipients/message. For the May data, recipients/connection = 1.22. It bottomed out at 1.10 in March, and has been slowly rising ever since.

Because of this, we have had the multiplier set to 1 since February (which is when we started running the script).

Okay..here’s my thought. 3 is way too high, we recently changed to 1. I think the multiplier is just a marketing tool for Senderbase. Not that is a bad thing but trying to convert connections to messages, may sounds interesting but I don’t think it’s the right thing to do, the system never processes the messages so I don’t think it should be reported that way. Connections are connections…messages are messages. I feel it’s one of those things that you may never find the correct data, there are too many variables.. For example what if you have multiple Ironports/Listeners listed in your MX record. There is no finality to a blocked IP, it will try to send the same emails again…so is it proper to report a dropped connection with 3 recipients but gets blocked by 4 of your MTAs as 12 messages or 4 for that matter? If you would have accepted it, wouldn’t have only been 3 messages. But if you count them as straight connections dropped, it is undisputedly 4. No matter how many times it tries counting each connection as 1 is a true report of connection attempts.

Senderbase is too good of a tool to have to fluff it up or convert to messages.. It’s a connection management tool, probably the best I have seen. It rids us of unwanted “Connections” so we can process the accepted “messages” and connections more efficiently. Why not report it that way?

For example what if you have multiple Ironports/Listeners listed in your MX record. There is no finality to a blocked IP, it will try to send the same emails again…

For us, we have a single MX record that points to a load balancer that has connection persistence turned on. This prevents the "try all the MX servers until I find one that works" tactic, and thus eliminates that source of statistical inflation. However, in theory this only reduces rather than eliminates the problem, because the spammer could retry the same message to the same recipient later. But in practice, spammers don't retry. So although I fully agree with your "apples and oranges" argument, I'm also reasonably satisfied with what I'm getting now that I've set the multiplier to 1.

[SenderBase] rids us of unwanted “Connections” so we can process the accepted “messages” and connections more efficiently.  Why not report it that way?

My guess is because PHBs don't understand the difference between connections and messages, and simply want to know "how much spam did we save?" This gives them a useful, if admittedly somewhat flawed, estimate.

I think the multiplier is just a marketing tool for Senderbase.

I do think that a default of 3 is probably too high, and that marketing played a role in its choice. Jay Bivens almost says this himself (although I admit that I might be reading too much into his words):

So IronPort performed research and the numbers ranged from 1.8 to 2.8 recipients per TCP connection and so the multiplier of 3 was created *in order to demonstrate the value and actions of Senderbase Reputation Filtering.* [emphasis added]

( 1.8 + 2.8 )/2 = 2.3, so 2 would have been a better choice assuming an even distribution.

Even if marketing did play a role in that choice, in the grand scheme of things it's just not that big a deal. The setting is configurable, and that's good enough for me. Besides, if you really want the connections versus messages breakdown then just set the multiplier to 1 and read "[Messages] Stopped by Reputation Filtering" as "Connections Rejected by Reputation Filtering." You have to subtract that number from Total Threat Messages, of course, but that's what calculators are for. :)