Site-to-Site VPN: Manual IPSec

Despite residing in the heart of Silicon Valley here in California, I have exactly one ISP offering speeds greater than 25 Mbps - Xfinity. For the last few years, I had been running shop with a 100 Mbps down / 5 Mbps up cable plan (which Xfinity has graciously upgraded recently to 400 Mbps / 20 Mbps). As mentioned in the previous section, I do run a relatively heavy network, thanks to the lab infrastructure in place for evaluating systems and storage devices in addition to serving the needs of a typical family of four.

Back in India, there is a lot more competition among ISPs to serve consumers. There are multiple fiber-to-the-home (FTTH) service providers with a wide variety of symmetric speed options. For the new home, my parents opted to go with Airtel's symmetric 100 Mbps plan (costing approximately US $12 inclusive of taxes). Their network demands were not too heavy - a smart TV, couple of mobile phones, a desktop, and a notebook - with only a couple of clients being simultaneously active.

While I had multiple VLANs at home, with a specific subnet for guests isolated from the rest (automatically created when a guest Wi-Fi network is configured), the configuration for the UniFi Dream Machine had only one primary network and another guest network.

During the initial configuration of the UniFi Dream Machine, Airtel had provided a public-facing WAN IP for the UDM to pick up. There was a necessity to call up the ISP to put their gateway (FTTH terminal / Wi-Fi AP) in bridge mode, but that is outside the scope of this article. The UDM was configured with the appropriate credentials to authenticate over PPPoE and pick up the WAN IP via the bridge connection. With the public-facing WAN IPs of both sites at my disposal, configuring the site-to-site VPN was a breeze.

On the US side, activating the site-to-site VPN network creation prompted for the required details - network name, VPN protocol, the pre-shared key, and the server address. The USG Pro 4 supports manual IPSec and OpenVPN, with the former capable of getting hardware-accelerated. A random pre-shared key can be generated and copied over. The server address was set to the WAN IP of the USG Pro 4. Under the 'Remote Device Configurations' section, it was required to specify the remote subnets desired to be made visible locally, along with the WAN IP of the UDM.

Similar information was entered in the UDM, with the pre-shared key generated on the USG Pro 4 placed in the PSK field. Ubiquiti has slightly reworked the UI in the UDM's Network application (Network 7.2.94 vs. 7.1.68 in the USG Pro 4), with the 'server address' tag being replaced by 'UniFi Gateway IP', making things slightly more user friendly. The remote device configuration section is filled with the required subnets from the US side, along with the USG Pro 4's WAN IP.

Upon adding the new VPN network on both ends, there was a handshake between the two devices and I was able to access the devices in the Indian network from the US and vice-versa. The web UI configuration transparently handles all the port openings required on either end.

The site-to-site VPN setup was further augmented with an old NUC connected to the UDM. The PC was set up to run a squid proxy server. In the US, an Android tablet was dedicated to accessing the Indian OTT services and set up to access the Internet using the NUC as a proxy. This configuration worked fine for more than a month. There were a few interruptions due to power failures and DHCP WAN IP changes on the UDM side. The latter had to be reflected in the site-to-site VPN setup and resulted in some downtime, but was not a cause for major concern.

I was fairly happy with the setup and would have left it as-is, if not for waking up one fine morning and finding the VPN link down. Expecting the customary WAN IP change, I fired up the UniFi Network app and tried to figure out the new IP assigned to the UDM.

Using the 100.107.xx.xx IP in the site-to-site setup was not helpful in re-activating the VPN link. Given my lack of formal network administration skills, this ended up being my introduction to the nitty-gritty details of carrier-grade network-address translation (CGNAT) - a term I had only encountered in passing earlier.

Introduction The CGNAT Spanner in the Works
Comments Locked

35 Comments

View All Comments

  • Notmyusualid - Friday, December 23, 2022 - link

    Ridiculous comment.
    IPv6 is the bomb.
    Just because you have to learn something new, doesn't detract from its usefulness.
  • ballsystemlord - Saturday, December 24, 2022 - link

    Right! In facing the challenge to post on AT, I learned how to read through my messages to check them for spelling mistakes because there is no edit button. ;)
  • Skeptical123 - Thursday, December 22, 2022 - link

    ​​Sure, let's add another layer of abstraction/complexity to the Internet protocol suite (TCP/IP stack) . That's a great idea. ipv4 and ipv6 is not something one expects nor do most consumer deal with. They are however one of the more if not the simplest bit of said suite/stack.
  • lalagon - Wednesday, December 21, 2022 - link

    Updating the firmware on a ubiquiti product it's like trying a lottery ticket...
  • at_clucks - Thursday, December 22, 2022 - link

    Oh you can say that again... I only update when I can afford to lose the connectivity :D.
  • Samus - Thursday, December 22, 2022 - link

    Especially if you don't have the controller and java updated and the hardware adopted in the database. It's overcomplicated to the point I wonder why they set out to find a solution looking for a problem that didn't exist.
  • Seraphimcaduto - Sunday, December 25, 2022 - link

    Wow I thought it was just me being overly dramatic about the update process on my dream machine, glad I’m not the only one that thinks this. I love the uptime on my dream machine but the update process feels like a jump to light speed without a nav computer.
  • jhoff80 - Wednesday, December 21, 2022 - link

    For what it's worth, 'plain' Wireguard is supposedly available on the Dream Router, Dream Wall, and Dream Machine Pro SE (anything with OS 3.0). My regular Dream Machine Pro is still awaiting that update though, so I can't say for sure how well it works.
  • Maltz - Wednesday, December 21, 2022 - link

    I use UniFi switches and AP's, but I wouldn't touch their routers except for only the most basic consumer-level stuff that for some reason required remote management. (I run a USG at my mom's house.) Their camera systems are good, too, but they have a habit of pulling the rug out from existing users in their non-network product lines like phones and NVRs.
  • Seraphimcaduto - Sunday, December 25, 2022 - link

    I have their switches, APs and the Dream Machine and you’re right; the APs and switches are awesome but the router has some strange limitations. The uptime has been a lot better than most of the other consumer gear I’ve had though. if anyone has any better suggestion for a router, I’m all ears.

Log in

Don't have an account? Sign up now