Supermicro SuperServer E302-9D Review: A Fanless 10G pfSense Powerhouse

pfSense Configuration for Benchmarking

A perusal of the FreeBSD firewall performance evaluation guidelines and the accompanying infrastructure helped us narrow down the scope of testing. As elaborated in the section covering the testing methodology, the DUT was configured in various states and the iPerf3 regular TCP benchmark and the pkt-gen sweep for different packet sizes were run for traffic passing through the firewall. A test of the L3 forwarding capabilities of the DUT was also performed using the ipgen benchmark while keeping in mind its stimulus-generating machine limited nature.

The table above summarizes the different states of evaluation and the shell commands used to place the DUT in that mode.

The additional firewall rules for the PF (Default Ruleset) case (added using easyrule / firewall log view) are as below:
pass in quick on ixl2 inet from 172.16.0.0/24 to 172.16.1.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on ixl2 inet from 172.16.0.0/24 to 172.16.10.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on ixl3 inet from 172.16.1.0/24 to 172.16.0.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on ixl3 inet from 172.16.1.0/24 to 172.16.11.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on ixl0 inet from 172.16.10.0/24 to 172.16.0.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on ixl0 inet from 172.16.10.0/24 to 172.16.11.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on ixl1 inet from 172.16.11.0/24 to 172.16.1.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on ixl1 inet from 172.16.11.0/24 to 172.16.10.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on igb3 inet from 172.16.20.0/24 to 172.16.21.0/24 flags S/SA keep state label "USER_RULE"
pass in quick on igb2 inet from 172.16.21.0/24 to 172.16.20.0/24 flags S/SA keep state label "USER_RULE"

The contents of the /home/username/nat.pf file referenced in the PF (NATMode) row of the table are as below:
set limit states 100000000
nat on ixl0 from 172.16.0.0/16 to any -> ixl0
nat on ixl1 from 172.16.0.0/16 to any -> ixl1
nat on igb2 from 172.16.0.0/16 to any -> igb2
pass in quick all keep state
pass out quick all keep state

The IPsec evaluation doesn't follow the steps outlined for the other modes. Instead of using both the source and the sink, along with iPerf3 and pkt-gen programs running on either side, only the source and the DUT are used. A baseline iPerf3 run between the source and the DUT (with no IPsec communication) is used for comparison. The communication between the two sets of ports is configured for IPsec using the script template below (invoked from the shell as an argument to the setkey -f command). The previous security policies and associations are flushed prior to the invocation.
flush;
spdflush;
# Host to host ESP
# Security Associations
add 172.16.0.2 172.16.0.1 esp 0x10001 -E -A ;
add 172.16.0.1 172.16.0.2 esp 0x10002 -E -A ;
add 172.16.1.2 172.16.1.1 esp 0x10003 -E -A ;
add 172.16.1.1 172.16.1.2 esp 0x10004 -E -A ;
# Security Policies
spdadd 172.16.0.2 172.16.0.1 any -P in IPsec esp/tunnel/172.16.0.2-172.16.0.1/require;
spdadd 172.16.0.1 172.16.0.2 any -P out IPsec esp/tunnel/172.16.0.1-172.16.0.2/require;
spdadd 172.16.1.2 172.16.1.1 any -P in IPsec esp/tunnel/172.16.1.2-172.16.1.1/require;
spdadd 172.16.1.1 172.16.1.2 any -P out IPsec esp/tunnel/172.16.1.1-172.16.1.2/require;
The template above is for the DUT side, with the one on the source side being similar (the in and out are reversed in the security policies section).

The next section provides additional benchmark processing details along with the results for both iPerf3 and ipgen tests. That is followed by a discussion of pkt-gen benchmark results.