cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
7656
Views
25
Helpful
10
Replies

New FirePower 2100 performance Numbers

Wise_Man_1
Level 1
Level 1

Hi Experts,

I have received several questions from some customer regarding the new FP data sheets, and I am hoping to find some answers here,

First:

Here is Cisco official statement "From preliminary testing, we’re seeing minimal impact on large packet firewall throughput when enabling intrusion inspection, SSL decryption, and other functions. In fact, with IPS fully enabled, we see with large packets less than 1% throughput degradation to network traffic. Contrast that with the typical 50% or greater impact in competing designs" 

however in the Data Sheet it states on page 5 : Note: Performance will vary depending on features activated and network traffic protocol mix and packet size characteristics. Performance is subject to change with new software releases. Consult your Cisco representative for detailed sizing guidance.

which contradicts with the first statment

Secondly:

in the old data sheet , the FP4100 had a sizing throughput (with 450 Bytes packets) of 4 Gbps with everything enabled, in the new data sheet, it says that it remains at 10Gbps with all service enabled  @ 1024 byte packets, to be honest this is very troubling, as this means (by doing some simple math) that the performance would be 1.6 Gbps wit 200 bytes packets, and 700 Mbps with 100 bytes packets (does that make any sense?)

Finlay

Cisco Data Sheets mention packet size for performance testing, while competitive data sheets mentions 64K HTTP transactions, what exactly is the difference between the two

Thanks

Wise

10 Replies 10

Oliver Kaiser
Level 7
Level 7

Hi Wise,

I understand your confusion caused by cisco marketing foo. To start of with your first question surounding this statement:

1)

"From preliminary testing, we’re seeing minimal impact on large packet firewall throughput when enabling intrusion inspection, SSL decryption, and other functions. In fact, with IPS fully enabled, we see with large packets less than 1% throughput degradation to network traffic. Contrast that with the typical 50% or greater impact in competing designs"

This is marketing at its best.  I am not sure how the person who wrote this defines *minimal impact* but I  think is wrong. Sizing firepower solutions is still the same its just that they were conservative on the minimal throughput of the solution without avc & ips.

SSL Decryption will still cause a 80% performance hit since the crypto chips are not used yet and AMP will still cause a 30-50% performance hit.

2)

Smaller packets will always cause less firewall throughput. As a general rule of thumb 440/450 byte is a good representation of real world traffic. Keep in mind that throughput reduction is not linear, if you want to know how many percent performance degradation you can expect take a look at NSS Labs NGFW report which tests firewalls using 64,128,256,512,1024,1512 byte sized packets.

For IPS appliances 440 bytes are commonly used to measure performance, for classic firewalls 1024/1500 byte packets are mostly used. Since NGFW fit in both spaces the packet size used for performance numbers will be listed with higher packet size (e.g. 1024) to paint an accurate comparison to other vendors which also use 1024 byte sized packets for testing.

3)

The transaction size is the amount of data transfered during a single http session used for performance testing. From my understanding this value alone doesnt have much significance without knowing the packet size.

Jawad Al Akrabawi
Cisco Employee
Cisco Employee

Hi Wise,

64K http transaction is a false way of marketing done by a specific NGFW vendor to show higher throughput, they hide the fact that the packet size could be around 1500 bytes (max on ethernet) which is rare to encounter in real world traffic.

In real world network traffic, most packet sizes will be less than 500 bytes, especially for most of Internet traffic. Showing performance numbers in 1024 Bytes is better than showing numbers in 64k http, and showing numbers in 450bytes would be the best.

You can do some more research on "IMIX" traffic profile to have an idea how traffic is distributed in terms of packet size.

Regards,

Jawad

 

Hi Jawad,

Thank you for your response, I was looking for a common criteria to compare performance numbers of different vendors as per my customers request, and since I am a Cisco partner I came here :)

I know the difference between different traffic types and mixes (real world traffic, imix, emix ...etc) and the previous posted response and data sheet numbers were good enough for my customers but the problem I am facing now, is the latest NSS lab report performance number for the 4110 is 2.4 Gbps with a 75% degradation in the data sheet numbers which is way less than the competition, I realize that we came in the recommended quadrant, while others did not, but we were not the only ones there, so is there an official response to the latest NSS labs report ?

Thanks

Wise

Hi Wise,

All major NGFW vendors publish throughput in datasheet with:

- Minimal inspection.

- Based on 1024 byte or larger packet size.

The NSS labs tests were calculated as an average of real world traffic, probably around 540 Bytes packet size. Because of average packet size, traffic mix, and security policy used in the
tests, there is a difference in throughput.

The FP4110 is only one of 4 models, but in all cases, what does customers look for:

Cisco Firepower leads again : Outperforming eight competitors in security effectiveness; blocking 100% of evasions, and besting several major vendors by over 50 points.

Many other vendors were given the chance to correct their evasion deficiencies AFTER the testing, The corrections made by competitors only address the evasions that NSS highlighted, not core resistance to evasion techniques to cover the thousands that exist !

Two students take a course: one gets an “A” grade, and the other fails. The failing student (not Cisco) can retake the course after a re-study of the topic they failed in while the exam questions of the re-take were exactly the same.... Would you really trust student "B"?

Regards,

Jawad

Thank you Jawad for the clarification,

there is no doubt about the security effectiveness of the Cisco firewall, and it superiority over the others, my main concern is regarding the sizing and hence the question regarding the performance, other vendors has only a 20 - 25% degradation in performance while Cisco had a degradation of more than 75% .

the customer wanted a 10G firewall, so I was proposing the FP4110, but with 75% degradation, the customer is asking for the FP4150, and he is saying that it is still below his requirements, which will kick us out due to huge price difference, so if there is any official response from Cisco on the performance degradation compared to data sheet numbers i would be great

Regards

Wise

Hi Wise,

As mentioned earlier, there are a lot of parameters that affect the throughput, ranging from average packet size, traffic mix, tuning security policies and so on. If you look at the connections per second, concurrency per mbps or various HTTP response of the 4110, you will see that the 4110 outperforms others as a percentage of total claimed throughput. Generalizing a certain threshold without understanding the customer's traffic types and patterns could lead to over-sizing the solution.

Let's say that you tested the FP4100 with "connectivity over security" policy instead of the "balanced connectivity and security" policy, you will for sure end up with much higher throughput, but would your customer be happy about low security effectiveness? What if you let the FMC intelligently discover what relevant policies to be applied based on the customer's traffic?

In all cases, the FP4100 series is certainly positioned as a NGFW & NGIPS platform in the data center. Many other NGFW vendors do not even appear in Gartner when it comes to "Intrusion Detection and Prevention Systems" although they claim NGIPS capabilities.

Regards,

Jawad

Hi Jawad,

you are preaching to the choir, I have explained to my customer  the different traffic patterns and the multi-scale in assessing the performance of the firewall (throughput, CPS and CC), however his response was

" All vendor satisfy my requirements for new connections per seconds and the number of concurrent connections  with the proposed platform, that is why we are looking the the NSS labs for the performance testing, as it used the same test with all vendors, with different traffic mixes, Cisco scored the worst in terms of performance compared to the data sheet numbers ranging from  465 Mbps to 3.25 Gbps with an average of 2.45 Gbps for a firewall that is marketed as a 10 Gbps firewall, in terms of security effectiveness, even though Cisco scored the highest in the HTTP evasion techniques protection, they scored the worst in the exploit block rate"

so with this in mind, I ask again does Cisco have an official response to the NSS labs report to share with my customer, or does Cisco acknowledges that all the findings in it are correct?

Regards

Wise

Hi Wise,

I am not sure that your customer has the right reports referring to NSS labs HTTP evasions or exploit block rate. We were the only ones along with one other vendor actually who fully passed all NGFW tests and IPS tests including evasions and exploit block rates.

Are you quite sure that the customer is referring to NSS labs 2017? I would suggest you reach to the Cisco SE handling the customer account to share the reports.

The vendor you are competing with (supposedly being a Gartner Leader) actually scored at the bottom and were not recommended by NSS labs.

Regards,

Jawad

Hi Jawad,

yes the customer has the correct report, and my question here is not about the security effectiveness or  block rate, my question is about the performance numbers in the report, which I am starting to believe are true since no one at Cisco is disputing them and everybody is deflecting their answers to security effectiveness and block rate or avoid to answer all together.

the Cisco SE covering the account is not responsive, and is avoiding responding to to accepting to visit the customer.

I would like to thank you for your patience and resonances, but since nobody from Cisco disputed the performance numbers in the NSS labs, I will have to tell the customers that Cisco are not acknowledging these numbers to be true but they are not disputing them either (which indicates that they are accepting the results as is)

Regards

Wise

Hello sir,

 

I have the same issue.

Form the 2017 NSS  report downloaded from Cisco web site: 

https://www.cisco.com/c/dam/assets/offers/pdfs/NSS-Labs-NGFW-Comparative-Report-Security-Value-Map-SVM.pdf?oid=anrsc000899&elqTrackId=B685F60A2E5D08ECBCFCF7A98AAD510B&elqaid=5663&elqat=2

 

The performance rate for the FP4110 downgrades a lot, but all the others in the report don't.

If Cisco cannot have a good explanation about it, I hope Cisco can tell us the real performance about the firepower series NG FW.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Review Cisco Networking products for a $25 gift card