Strange DNS issue with guest network

  • 1
  • Question
  • Updated 1 year ago
We've had an ongoing problem with our guest network. Let me describe the network first, and then the problem.
We have 40+ AP330s with GRE tunnels to another AP330 for guest network traffic (the Guest Bridge). All are running HiveOS 6.5r4. The Guest Bridge is behind a firewall, has radios disabled, and eth1 is configured in Backhaul mode for the guest VLAN.

The guest SSID has a From-Access firewall policy.

Ordinarily the guest WiFi works as expected. When the problem occurs, up to 40% of the DNS query traffic to both local and remote DNS servers does not reach the switch on the eth1 side of the Guest Bridge but ping is unaffected. (We determined this by putting an ethernet tap between the Guest Bridge and the switch and sending a query at regular intervals to see gaps.) This obviously affects most internet bound traffic on the guest WiFi. (Wired traffic on the guest network has no such problems and DNS traffic on the corporate SSID has never been affected in this way.)

Things we have tried to resolve this (without success, all of these are still effective):
- Matching VLAN IDs between the Guest Bridge and the ports on the switch
- Remove ALG from the DNS rules in the From-Access firewall
- Upgrading to 6.5r4 (we had the problem on previous versions too)

We get temporary relief by hard resetting the Guest Bridge.

An aside:
In order to see if the DNS queries were entering the Guest Bridge, we captured the GRE traffic entering the Guest Bridge but Wireshark could not decode it. Can the GRE mode be changed to allow Wireshark to decode like in this Aruba (sorry!) article? https://community.arubanetworks.com/t5/Controller-Based-WLANs/How-can-we-see-the-packets-tunneled-in...

Suggestions are appreciated.

Cheers
Photo of Fraser Hess

Fraser Hess

  • 60 Posts
  • 7 Reply Likes

Posted 1 year ago

  • 1
Photo of Fraser Hess

Fraser Hess

  • 60 Posts
  • 7 Reply Likes
I've spent a bit of time of this today, looking at the tech data from the Guest Bridge, and I now think this is tied to running out of forwarding engine IP sessions. The occurrence of:
err     kernel: [fe]: exceed maximum number of IP sessions [8191] allowed, per 80000
in the log corresponds with the days we have had problems in the last 2 weeks.

Can I increase the number of sessions? Can the sessions age out faster? I'm not finding this in the help.
Photo of Eastman Rivai

Eastman Rivai, Official Rep

  • 146 Posts
  • 17 Reply Likes
You may need to check if there are wireless clients running peer to peer internet file sharing application, such as bit torrent.  You may need to block this kind of applications from accessing the internet as it uses lots of IP sessions.  You may also need to block multicast traffic to be forwarded across the GRE tunnel. You can configure this on the management option.

The number of IP-sessions cannot be increased however you can limit the number of IP-session per client. Regarding the session age, The IP-session will always be in the list as long as it has traffic.  it will be cleared 5 minutes after being idle by default. You cannot modify it unless you have a customised network service that is used in the firewall rule, you may then specify the idle time out.

I hope this answer your question.
Photo of Fraser Hess

Fraser Hess

  • 60 Posts
  • 7 Reply Likes

We resolved this by adding a 2nd guest bridge AP. Connections are pretty much load balanced between the two.