Thinking about Security Power…..

network

Thinking about Security Power…..

Check Point introduced a new metric called Security Power. They want to provide more useful information about the realistic performance of their appliance beside from “show off” numbers which claim high throughput.

This is a good and valid approach because max. throughput is measured with just one rule in the rulebase (any-any-any-accept) and the traffic consist only of 1500 byte long UDP packets. Plain and simple traffic, nothing really to do here for the firewall.

On the other hand we can have the worst case which would be 64 byte TCP packets. Here we have only small numbers when it comes to throughput, nothing worth to be found in a marketing slide.
But even this number is important as is defines to lower end of performance for a system.

Short story about this: once we had massive performance problems at a customer site. Some web-based application was running very slowly and the firewall was nearly at 100% CPU. We couldn’t find anything at first as the firewall was operating within normal parameters besides from high CPU load. Was this a bug in the software? A hardware failure? After some unsuccessful searching I did a traffic capture with fw monitor and analysed it with WireShark. The disposition of packet sizes showed me a very high appearance of really small packets. Looking further into the dump I realized that they derived from short LDAP queries that were going through the firewall. In terms of throughput is was not that significant. But in terms of number of packets it was significant. Gladly I had some information from inside Check Point about the performance of appliances in best and worst case scenarios. We could match the hardware from one of the appliances to the OpenServer system the customer firewall was running on, they were nearly the same. Then we compared the numbers for LDAP queries we had seen on the live system to the number of 64 byte TCP packets that could be handled by the appliance at best. And we found out that the 100% CPU load on the customer system was not coming from a software bug or hardware issue but from the firewall operating at the maximum performance it could deliver for that kind of traffic.

So coming back to Security Power: Check Point claimes that it is measured with a rulebase consisting of 100 rules which is fairly the amount of rules the normal customer has in it’s rulebase. The traffic used for measuring is a real-world traffic mix for whatever that means. Hopefully it’s mostly HTTP, some FTP, SMTP, SNMP along with DNS, NTP and ICMP. That’s what I would expect to be real-world traffic.

Based on the rulebase, the traffic-mix and the throughput needed a value is calculated called Security Power Unit (SPU). A UTM-1 3070 can deliver a max SPU of 298 whereas a Power-1 5070 delivers 596 SPU.

The online tool that will be available shortly takes into consideration which blades you use to calculate the SPU needed. In the example Check Point provided during the webcast the key numbers were 200 Mbit/s max throughput with three blades enabled (FW, IPS, AC). The traffic type was “Internet” for whatever that means. Maybe they take into account that different real-world traffic profiles arise from the use of the Security Gateway at the perimeter or internally. When deployed internally I would expect more SMB traffic for example. The tool calculated that 205 SPU were needed to fulfill the requirements.

Let’s compare the numbers: 205 SPU are needed for 200 Mbit/s real-world traffic with FW, IPS and AC. The appliance UTM-1 3070 is capable of nearly 300 SPU, so the hardware will be about 66% loaded.

In the pricelist you will find that the max throughput measured the old way is 4.5 Gigabit/s. So assuming a linear progress in CPU consumption linked to traffic this means that a UTM-1 3070 with 66% load is capable of 2.97 Gbit/s max throughput.

FW with one rule = 2.97 Gbit/s
FW with 100 rules, IPS and Application Control = 200 Mbit/s.

Really different numbers, aren’t they?

Some CCIE told me that with Cisco Routers he divides the performance numbers given by Cisco by half for every feature he implements. Using NAT? 50% performance left. Using ACL? 25% left. Using VPN? 12,5% performance left. Meaning you go from 1 Gbit/s routing performance to 125 Mbit/s while using 3 additional features.

This show that it’s not only Check Point that has only marketing numbers out there while real-world performance is something completely different.

So I appreciate the approach of Check Point to give us something to choose the right Security Gateway for the desired environment and it’s needs. On CPX someone from Check Point whom I trust told me that the tool is really accurate.

It will be interesting for me to check if I chose the right appliances in the past according to the appliance selection tool.

But there’s still one thing missing with Security Power: the ability to measure also OpenServer systems.
I know that Check Point is focusing on appliances, but there are still good reasons to deploy OpenServer hardware instead of appliances. You can’t expect Check Point to deliver SPU numbers for all server / NIC combinations found in the HCL, as CPU, memory as harddrives differ from system to system and each component has influence of the performance. But some testing tool to measure the max SPU of an OpenServer would be really great. I would guess that SPU is calculated by something like instrustions per second on specific hardware and that Check Point knows how much computing power is necessary to process a certain amount of traffic with selected blades. In this case you can transfer this into the OpenServer world.

Is this likely to happen? I guess not. Check Point want’s to sell appliances instead of licenses, judging by their recent activities.

So we still have to stick with our appliance hardware list that can be found on this blog. And by comparing appliance hardware with your OpenServer you can estimate the performance that this system is capable of.

Also on CPX I learned that Check Point is not happy about my hardware list for the appliances. They prefer to talk about performance numbers instead of hardware that is build into an appliance. I can totally understand them – but please try to understand our position as technical people. First of all we’re curious and we CAN find out what hardware is in the box…. and therefore we will DO it
Second we need to choose the right solutions for our customers day by day. And when it comes to real world security requirements and traffic mix the Check Point performance numbers or far from being realistic. Wonderfully shown by Check Point itself in the example above with the UTM-1 3070 appliance were the tool calculates that a fully loaded appliance will only deal with 300 Mbit/s instead of 4.5 Gbit/s under realistic conditions. So we compare the appliances, were we have Security Power numbers and a tool, to OpenServer hardware based on the components inside the appliances.

Check Point, please understand: we need to know! Appliances are only 65% of real customers environments, the rest is OpenServer and we as CP partners have to cover this as well. Please give us a benchmark tool for measuring SPU on OpenServer!

And I would also prefer to have the number of max throughput with 1500 bytes UDP packets as well as 64 byte TCP packets for the appliances. This helps us also to get the right solution / core license.