How to configure CVG NOT to NAT all client traffic

  • 1
  • Question
  • Updated 5 years ago
  • Answered
Ok, this one has me stumped. Our current setup:
* We are in essence an ISP/datacenter with a public Class B IPv4 address space
* HiveManager VA hosted on our Cisco UCS (pubIP)
* a HiveOS VA in another VM acting as a CVG, configured with
* 4 AP330s we use in our building
* 1 BR200-WP we're trying to configure for a remote site
* all running v6.1r1

HM, HiveOS VA, and APs all work fine. The BR200-WP is configured with a Comcast Business Internet static IP address, whereas all other gear has addresses in our Class B space. The BR200-WP is hooked into HM and all is configured properly and showing that the VPN tunnel is up between the BR200-WP and CVG. So we have connectivity.

The remote site is our director's, and we've assigned them a /27 of their own in our public space. The CVG is configured with just its WAN interface to have a public IP, and we have routing statements set on the router beyond the CVG to send all traffic destined for that /27 to the CVG.

Now here's where things are weird. Devices at the remote site get IPs as they should (public IPs in the /27 if they're plugged into the BR200-WP ethernet ports or on the staff SSID, private addresses in the 192.168. space NAT'd to the Comcast static IP and shunted straight to the Internet if they are on the guest SSID).

HOWEVER, when we test from a PC at the site and visit a site like www.ipchicken.com, the IP address that the PC appears to be sourced from is the IP address of the WAN interface of the CVG!!! So it's as if the CVG is NATing all traffic! This is true whether the destination site is outside our Class B (e.g., www.ipchicken.com) or a server we run to do the same that's within our Class B.

Can anyone tell me how to turn this off? I can't seem to find it anywhere. I would like that the PCs at the remote site source from their own IPs to wherever they go, and that all traffic from the staff LAN be routed through the VPN tunnel. (Oh, in that regard, I have a routing policy configured to do just in the network profile under 'Additional Settings', then 'Router Settings', with (*) Tunnel All set for the routing policy.)

Now note we can use a program like TeamViewer to remote into PCs at the remote site, so it's not like we don't have connectivity. But the side effect of all remote traffic being NATed by the CVG to its WAN IP is that it is impacting certain services due to the fact the endpoints aren't showing up with their own IP addresses.

If anyone can point me in the right direction for this, or even just explain to me HOW it is that the CVG is doing the NATing at all (vs. the BR200-WP NATing traffic with its WAN interface IP... namely, the Comcast IP), I am all ears.
Photo of Frank

Frank

  • 15 Posts
  • 8 Reply Likes
  • extremely frustrated

Posted 5 years ago

  • 1
Photo of Patience

Patience

  • 61 Posts
  • 0 Reply Likes
In our network, I have not even use any Routing Policy at all.

I have use private IP addresses 10.102.0.0/16 (Further subnetted /24 for each BR200) for Staff to tunnel inside the network and while configuring Network , I have chosen Internal Use at Network Type.

I have use another set of private IP addresses 172.16.200.0/24 for Guest users and chosen Network Type: Guest Use. All BR200 site will use same IP as it never traverse to CVG anyway.

I have use 172.19.0.0/16 for management and further subnetted for 512 branches.

So now in my hive manager, I can see my one of the BR200 interface ip address 172.19.3.1, External ip address 99.99.99.69 (AT&T uverse), and my pc at BR200 staff wifi is 10.102.12.16/24.

Now whenever I go to public site I will be Nated on 99.99.99.69, and still can get into my HQ network using same public IP which encapsulate my internal private IP assigned by CVG 10.102.12.16.

In conclusion, you do not have to use any of your Public IP at your remote site as you are already using comcast. And may not need to use routing policy.
Hope this works for you as it is working for us.
Photo of Frank

Frank

  • 15 Posts
  • 8 Reply Likes
Dhiraj,

Thanks for replying. And it's clear you're following the stock Aerohive configuration for remote sites as they outline in their documentation and courses like the ACNP which I took recently. Unfortunately, that's not how our network is configured. We own a proper IPv4 Class B address space, and while I understand the necessary evil of NAT/PAT, in general I do not subscribe to the "security through obscurity" approach. NAT/PAT is the bane of networking, as it introduces all manner of problems and the need for unnecessary workarounds.

More importantly in this case, though, is that we have an existing network infrastructure. The remote site HAS a public IP subnet. It is currently configured using a transport DSL configuration. This means the circuit connects the remote site directly into our core. That DSL circuit, however, has been unreliable at best due to carrier problems beyond our (and apparently their own) control. So we've had an alternative WAN circuit put in.

This new circuit is a cablemodem setup basically, like you might have at home, only business class. Unfortunately, there's no way to run Layer 2 traffic direct from the remote site back into our core. The carrier (Comcast) provides Layer 3 service. This means the remote site gets a Comcast IP on their WAN interface. So the end goal is to build a VPN tunnel from the remote site to our core. We tried that using the existing Cisco 877W (a Frankenstein box in my opinion that smashes a router with an ATM (DSL) interface and a Cisco WiFi AP into a single physical box, where each has its own IOS image, its own configuration file, and basically is a royal p.i.t.a. to configure/maintain). However, we have had various issues in getting the unit to function properly. I've gotten it to build a VPN tunnel back to our core, and to route all traffic over the VPN back to the core, but no matter how we configure things, the Cisco unit will not function properly outside our Class B space. Believe me, I spent a lot of time on this, as I figured an IOS device is an IOS device. But time and again when the configuration indicated it should work, it did not.

So having HiveManager and AP330s and a HiveOS VA already, we thought it was a good time to pitch Aerohive for branch sites, and this was a perfect opportunity to demonstrate how easy it would be. Unfortunately, while I suspect if you go with the stock examples Aerohive provides you'll do fine, for those of us who work with public IPs at our remote sites, it doesn't seem to be that simple (though in my book it should be).

Hence the question on this forum. So sorry, but what you're doing doesn't work for me at all.

Now, regarding your statement "Now whenever I go to public site I will be Nated on 99.99.99.69", that's what one would expect; namely, that you are NATing AT the BR200 using ITS external (Eth0/WAN) IP address. In our case, that's not what we're seeing. Again, for argument's sake, let's say our public IPv4 Class B network is the 1.2.x.x/16 network. The remote site LAN has 1.2.21.0/27 allocated for the PCs there. The BR200 is configured with a static IP on its Eth0/WAN interface of 5.6.7.8. The HiveOS VA sits back at our core on 1.2.3.4/24, and our HiveManager sits as 1.2.1.3 (again, these are made up IPs).

The BR200 "phones home to mommy" from 5.6.7.8 to 1.2.1.3 and builds the VPN tunnel, no problem. The BR200 acts as DHCP server and hands out addresses to the PCs at the remote site. So one PC might get 1.2.21.9. If you bring up a web browser and visit http://www.ipchicken.com, you might expect to see EITHER 1.2.21.9 if the PC shows up as itself (meaning the data went through the VPN tunnel) or 5.6.7.8 if the BR200 is NATing the remote site traffic directly to the Internet. What you would NOT expect to see is that the remote PC appears on ipchicken.com to be 1.2.3.4 (the IP of the HiveOS VA)! Yet THAT is exactly what we are seeing.

Why architecturally you would ever even WANT this to happen is beyond me. It means you'd have all of our remote site devices popping out on the Internet from a single IP address (i.e., PAT, not even NAT). That in turn means you've just choked your remote site traffic to a theoretical max of 65K unique connections, as there are only that many ports possible from a single IP in IPv4 (hell, in IPv6 as well).

But that is why I'm posting. If anyone else is seeing this, either because they wanted it that way or not, to find out how to turn that off. I want plain vanilla routing. I want a remote site to tunnel LAN traffic through a VPN back to the core and out, using simple, public IPv4 addresses. No NAT. No PAT. No fuss, no muss. This is network engineering at its simplest. Yet it's not how things seem to be working and it's frustrating.
Photo of Patience

Patience

  • 61 Posts
  • 0 Reply Likes
I am not familiar with other firewall lot but we are using Cisco ASA and it is easy to by pass NAting as well. I think you can also do No-NAT at the device where CVG is connected for all your public IPs in inbound direction. In our case I have done NAT 10.102.0.0 with same subnet 10.102.0.0 which keeps IP the same whenever remote client encapsulated with IPsec vpn tunnel.
I think, you simply try first changing your routing policy from Tunnel All with Split Tunnel and see what happens. It should route all web traffic using Comcast public NATed ip and tunnel internal to CVG. I also do defining all internal subnets at CVG.

Good luck.
Photo of Frank

Frank

  • 15 Posts
  • 8 Reply Likes
Dhiraj,

Again, our setup is quite different. We are using only public IP addresses. We are not doing NAT. We do not HAVE a firewall, Cisco ASA or otherwise, in the path from the remote site PCs to the Internet. It's PC with public IPv4 connected to BR200-WP connected via VPN tunnel to HiveOS VA running on a Cisco UCS connected to our core router. That's it.

So there's no "bypassing NAT", no "do No-NAT", or anything of the sort. The issue is that the HiveOS VA, the end of the VPN tunnel, which has a public IPv4 address (so that BR200-WP can actually connect to it to build the tunnel), is apparently NATing ALL CLIENT TRAFFIC TO ITS WAN IP ADDRESS. So there is NO OTHER GEAR INVOLVED EXCEPT AEROHIVE.

With regard to Tunnel All vs. Split Tunnel, for the internal LAN at the remote site, we want ALL traffic to traverse the tunnel regardless of final destination.

My question to the community is whether 1) they have ever seen this before, 2) why anyone would WANT to do this, and 3) most importantly, how to turn it off.
Photo of Frank

Frank

  • 15 Posts
  • 8 Reply Likes
Ok, I've had a ticket open with Aerohive and found the answer. Let me copy/paste the most relevant info here so others may find it.

It appears that NATing remote site traffic at the HiveOS VA (CVG) is BY DESIGN based on what I found in the online HiveManager documentation on the page at Configuration > VPN Services > VPN Service Settings, located under Layer 3 IPsec VPN Tunnels | Basic VPN Service Settings:

[NOTE: ALL CAPS is my emphasis]
________________________________________
To define a Layer 3 IPsec VPN services profile that makes use of all the default settings, choose the VPN gateway and define its external IP address, configure the default routing policy and any policy exceptions, and then click Save.

Layer 3 IPsec VPN: (select)

In the VPN Gateway Settings section, enter the following, and then click Apply:

VPN Gateway: Choose the name of a VPN gateway from the drop-down list. The VPN gateways that appear in this list are HiveOS Virtual Appliances that have been configured as Layer 3 VPN gateways and are not being referenced by any other VPN services profiles. You can configure a HiveOS Virtual Appliance as a Layer 3 VPN gateway on the Configuration > Devices > All Devices page or on the Configuration > Devices > VPN Gateways page. (For Help, see "VPN Gateway Settings".)

External IP Address: Enter the IP address that routers/VPN initiators must use as the termination point of their VPN tunnels. If the VPN gateway is behind a firewall using NAT, enter the public IP address on the firewall that maps incoming IKE and NAT-Traversal traffic to the WAN IP address on the VPN gateway inside the firewall. If the WAN interface has a public IP address and there is no NAT device between it and the public WAN, enter the WAN interface IP address for VPN gateway.

* IF THE EXTERNAL IP ADDRESS IS THE SAME AS THAT OF ITS WAN INTERFACE, THE VPN GATEWAY AUTOMATICALLY PERFORMS NAT ON TRAFFIC THAT IT RECEIVES THROUGH TUNNELS BEFORE ROUTING IT TO WEBSITES ON THE INTERNET, TRANSLATING THE SOURCE IP ADDRESS TO THAT OF ITS WAN INTERFACE. If the external IP address is different from that of the WAN interface, it does not perform NAT on outbound traffic to websites. It is assumed that when the WAN interface and external IP address are different, there is another device such as a firewall that will do the address translation before routing the traffic to the Internet.
________________________________________

After much back/forth, this morning our Aerohive Systems Engineer, Ben McDonald, wrote me of a solution in the interim.

As of now (we're running v6.1r1), it seems that there is no GUI option for disabling this NATing at the CVG. However, it IS possible to disable it via the CLI on the HiveOS VA.

The command you are looking for to override this default NAT behavior is the following:

no interface eth0 mode wan nat

That's it. Do note that whenever you update the configuration, you should use the compare with last configuration option. As of right now if you push a Complete or Auto configuration change, this setting will be removed

And according to Ben, there is already a Feature Request in to have this functionality added.

So for anyone in the same boat as myself, hopefully this post will come in handy.

To everyone at Aerohive and on this forum who took the time to read/reply/help, thank you. You guys rock.
Photo of Mike Bradshaw

Mike Bradshaw

  • 10 Posts
  • 4 Reply Likes
Thank you for explaining this and not only how, but why. Several items in your thread here were of use to me. Kudos and thanks!
Photo of Frank

Frank

  • 15 Posts
  • 8 Reply Likes
Oops, forgot to set my emoticon...
Photo of Amanda

Amanda

  • 396 Posts
  • 25 Reply Likes
Thank you for the update Frank. I'm so glad you got your problem resolved and that you shared the solution with the community.