Site-to-Site VPN: Manual IPSec

Despite residing in the heart of Silicon Valley here in California, I have exactly one ISP offering speeds greater than 25 Mbps - Xfinity. For the last few years, I had been running shop with a 100 Mbps down / 5 Mbps up cable plan (which Xfinity has graciously upgraded recently to 400 Mbps / 20 Mbps). As mentioned in the previous section, I do run a relatively heavy network, thanks to the lab infrastructure in place for evaluating systems and storage devices in addition to serving the needs of a typical family of four.

Back in India, there is a lot more competition among ISPs to serve consumers. There are multiple fiber-to-the-home (FTTH) service providers with a wide variety of symmetric speed options. For the new home, my parents opted to go with Airtel's symmetric 100 Mbps plan (costing approximately US $12 inclusive of taxes). Their network demands were not too heavy - a smart TV, couple of mobile phones, a desktop, and a notebook - with only a couple of clients being simultaneously active.

While I had multiple VLANs at home, with a specific subnet for guests isolated from the rest (automatically created when a guest Wi-Fi network is configured), the configuration for the UniFi Dream Machine had only one primary network and another guest network.

During the initial configuration of the UniFi Dream Machine, Airtel had provided a public-facing WAN IP for the UDM to pick up. There was a necessity to call up the ISP to put their gateway (FTTH terminal / Wi-Fi AP) in bridge mode, but that is outside the scope of this article. The UDM was configured with the appropriate credentials to authenticate over PPPoE and pick up the WAN IP via the bridge connection. With the public-facing WAN IPs of both sites at my disposal, configuring the site-to-site VPN was a breeze.

On the US side, activating the site-to-site VPN network creation prompted for the required details - network name, VPN protocol, the pre-shared key, and the server address. The USG Pro 4 supports manual IPSec and OpenVPN, with the former capable of getting hardware-accelerated. A random pre-shared key can be generated and copied over. The server address was set to the WAN IP of the USG Pro 4. Under the 'Remote Device Configurations' section, it was required to specify the remote subnets desired to be made visible locally, along with the WAN IP of the UDM.

Similar information was entered in the UDM, with the pre-shared key generated on the USG Pro 4 placed in the PSK field. Ubiquiti has slightly reworked the UI in the UDM's Network application (Network 7.2.94 vs. 7.1.68 in the USG Pro 4), with the 'server address' tag being replaced by 'UniFi Gateway IP', making things slightly more user friendly. The remote device configuration section is filled with the required subnets from the US side, along with the USG Pro 4's WAN IP.

Upon adding the new VPN network on both ends, there was a handshake between the two devices and I was able to access the devices in the Indian network from the US and vice-versa. The web UI configuration transparently handles all the port openings required on either end.

The site-to-site VPN setup was further augmented with an old NUC connected to the UDM. The PC was set up to run a squid proxy server. In the US, an Android tablet was dedicated to accessing the Indian OTT services and set up to access the Internet using the NUC as a proxy. This configuration worked fine for more than a month. There were a few interruptions due to power failures and DHCP WAN IP changes on the UDM side. The latter had to be reflected in the site-to-site VPN setup and resulted in some downtime, but was not a cause for major concern.

I was fairly happy with the setup and would have left it as-is, if not for waking up one fine morning and finding the VPN link down. Expecting the customary WAN IP change, I fired up the UniFi Network app and tried to figure out the new IP assigned to the UDM.

Using the 100.107.xx.xx IP in the site-to-site setup was not helpful in re-activating the VPN link. Given my lack of formal network administration skills, this ended up being my introduction to the nitty-gritty details of carrier-grade network-address translation (CGNAT) - a term I had only encountered in passing earlier.

Introduction The CGNAT Spanner in the Works
Comments Locked

35 Comments

View All Comments

  • Zoolook - Monday, December 26, 2022 - link

    Dead? Yeah sure.
    It's only up to 40% of users now on a global scale and the increase is a steady 5% (of total users) y/y increase, so I'm sure it's about to roll over and go away any time now.
    Just because your ISP is shielding you so far doesn't mean it's not there.
  • PeachNCream - Monday, December 26, 2022 - link

    Don't buy into the hype kids. Ubiquiti's best selling point is that it looks pretty when its sitting on the rear seat of your car as you drive it to the nearest electronics recycling facility.
  • egc - Tuesday, December 27, 2022 - link

    Both DDWRT and OpenWRT support Wireguard (and of course OpenVPN) it is very easy to setup a site-to-site setup.
    Runs on relative cheap hardware e.g. NetGear R7800 with WireGuard speeds of about 300 Mb/s
  • Threska - Wednesday, December 28, 2022 - link

    Asus-merlin does as well. Having a choice helps with different situations.

    https://www.top10vpn.com/guides/wireguard-vs-openv...
  • zanon - Saturday, December 31, 2022 - link

    Ubiquiti sadly become a development dumpster fire and been going downhill internally for a while. I really REALLY wouldn't recommend them for a greenfield deployment these days. In particular their gateway/services has sucked badly for a long time and I dumped it for OPNsense a few years ago, now starting the move to Omada for switching/WiFi. Wireguard is excellent and reliable, though a lot of people may find Nebula a better match since while still using Noise it does true meshing. A fixed IP (a small light VPS will work fine) is only necessary to help clients find each other and coordinate, then the clients talk directly after. Still fully self hosted, no 3rd party service needed.

Log in

Don't have an account? Sign up now