Creating a mesh environment - need advise

  • 1
  • Question
  • Updated 2 years ago

I am looking to get some advise here. I am trying to set up mesh links between my outdoor APs for redundancy.

The plan is to use the fibre connections as main links and use the wireless links as backup.
All Layer2 - 5GHz radio for mesh / 2.4GHz for clients. All within one hive.
I am using HiveManager NG.

I tried creating a mesh but I failed.
I have set up new TEST policy, added 2 AP1130's. 5GHz radio on both are set up as Mesh.
They are sitting 10m away from each other in case anyone asks.

However they don't establish mesh at all. Well, I am not entirely sure to how to check it.

Is there any Aerohive CLI command reference available? Any other document that would help me?

I have attached the image showing what I am trying to achieve.

Any assistance will be appreciated.

Photo of Kamil Słodecki

Kamil Słodecki

  • 4 Posts
  • 0 Reply Likes

Posted 2 years ago

  • 1
Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
A few things.  If the APs are wired to a switch, they will default to it for backhaul and not even try to mesh across.  You will need to shut off the link on the ethernet port (but keep POE on).

A command to see mesh connection/status is "show hive <your hive name> neighbor

It might benefit you to statically assign the radios to a set channel as well to speed up the mesh establishment.  If you need link passed through, for production, you will have to set the eth0 interface for the AP to Bridge-Access or Bridge-.1Q (depending if you need to tag VLANs across the link).
Photo of Kamil Słodecki

Kamil Słodecki

  • 4 Posts
  • 0 Reply Likes
Hi Brian,

Many thanks for the prompt response. 
Next question is, how do you bridge buildings with 1130's if you have switch behind? 
Keep in mind that Aerohive has the high gain 5GHz directional antenna which I believe is designed to bridge the buildings?

How do you connect the rest of the network if you shut off the eth port? 

Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
Disabling the ethernet port is a temporary fix to get the two to link up if they are already deployed.  You will need to set the APs ethernet port to Bridge-Access or Bridge-.1Q for it to not use it as its sole backhaul link (which is does by default if it sees link on it).  Once you re-configure the AP ethernet port or pre-configure the AP ethernet port, you will not need to keep the switch ethernet port disabled.  
Photo of Kamil Słodecki

Kamil Słodecki

  • 4 Posts
  • 0 Reply Likes
I should have mentioned that I had both eth ports configured as dot1q already. 
I have changed the eth to backhaul and back to trunk and seems like the mesh in now on.

Mac Addr       Chan Tx Rate Rx Rate Pow(SNR)     A-Mode   Cipher Conn-Time   Hstate Phymode Chan-width Hive-------------- ---- ------- ------- -------- ---------- -------- --------- -------- ------- ---------- ----
f09c:e979:0ea0  100    156M    156M  -47(48)           open aes ccmp  00:15:09     Auth    11ac      20MHz 

1. With mesh ON I keep getting this message from both AP's every 1-2min

2016-06-05 13:07:16 err     ah_dcd: ACSP: unable to find portal AP, rescanning.: Invalid argument

The portal AP term rings a bell but I cannot remember what it was. I believe it was something to do with the eth config? 

2. I am unable to set the channel manually on 5GHz radio. Only auto is an option to select. Works fine for 2.4GHz. 
What's weird here - 5GHz radio profile for my 230's and 390's gives me an option to choose between 4 channels only - 36,40,44,48 (not sure why only 4?). The 1130's show no channel to select at all.
Any ideas?
Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
So your link is up, which is good I take it?

As for the message you receive.  I suspect it may have to do with the APs set to auto channel selection as thats what Aerohives ACSP is - Automatic Channel Selection Protocol.

As for why there is no option to set static channels or limited static channel availability, what country code are the APs set to?  A "show boot-param" will show you the country code and region:

Region Code:        FCC

Country Code:       840

You can also try manually setting the channels from the CLI or from the Supplemental CLI.

int wifi1 radio channel <channel number>
Photo of Roberto Casula

Roberto Casula, Champ

  • 231 Posts
  • 111 Reply Likes
Kamil, if your radio profile does not enable support for DFS then you will only have the U-NII-1 channels (46, 40, 44, 48) available. U-NII-1 is for internal use only, so because the 1130 is set up as an external AP, there are no channels available for it to use without DFS enabled. Exactly which channels are allowed depends on your local country regulatory restrictions.

Putting that aside, Aerohive's mesh functionality is designed to extend wireless backhaul capability to access points where you cannot implement a wired backhaul (e.g. because of distance or other physical limitations).

What you are trying to do is to use the mesh capability to provide a wireless bridge link, which is a different scenario.

Aerohive's forwarding engine works similarly to a layer-3 routing protocol but at layer-2. Each AP's layer-2 forwarding engine maintains a forwarding table associating the MAC addresses of hosts with an interface through which that host is reachable. In a normal AP situation, this works as follows:

For each host which is connected to the AP wirelessly via a normal SSID connection, an individual MAC address entry is maintained. In addition to this, a default route is maintained via the backhaul interface to reach "the rest of the network". This basically means that each AP only needs to know how its locally connected hosts can be reached and assumes everything else is reachable via the backhaul.

Ordinarily, the backhaul would be via the wired Ethernet interface, but in the case of a meshed connection, that backhaul would be via a wireless backhaul interface instead. The AP which is upstream over this backhaul connection then maintains forwarding information for all the hosts it learns over this backhaul connection and then itself has its own default route either back over a further wireless backhaul connection to yet another AP and so on down the line until eventually we hit the wired Ethernet backhaul.

The more hops you have in your wireless mesh, the greater the number of individual MAC entries the APs in the mesh have to maintain (the number of entries increases the further back into the backhaul you go) - eventually you will hit a hard limit on the size of that table, and this is an important scaling limitation of all wireless mesh solutions.

The scenario you describe is much more complicated than this.

First of all, consider the situation where you ONLY had a single switch/AP on the right-hand side (rather than the three shown in your diagram) and for the time being let's look only at the wireless link (i.e. there is no wired connection). In this case, you would configure the Ethernet connection on the AP on the left-hand side of your diagram as a normal backhaul connection. That AP would have a default layer-2 route to the rest of the network via this backhaul link. The AP on the right-hand side of your diagram though, cannot be configured in the same way - it would need its default route pointing across the wireless link rather than on its Ethernet interface, so you would configure the wireless interface as "Backhaul" rather than "Access". The Ethernet interface on this AP then needs to be configured as a BRIDGE interface (type Bridge-Access or Bridge-802.1Q, depending on whether the interface is carrying just a single VLAN or multiple VLANs). This interface will perform MAC learning for the hosts on the Ethernet network. Depending on how many hosts you have there, you may hit architectural limits on table sizes in the forwarding engine, but if the number of hosts is relatively small, then you can get away with that.

This generally works OK, but there are a few problems from a wireless perspective (which aren't uncommon generally for wireless bridge links).

Using the 5GHz radio for backhaul in an outdoor environment, means you're going to need to use DFS. Even if you fix the channels at each end of the link (which you really need to do to avoid the APs having to "hunt" for one another across all channels), if DFS detects a possible radar signal, the AP MUST immediately switch to another channel at random. It then becomes a problem for the two APs to "find one another" again. Using the 2.4GHz radio for backhaul avoids this issue, but then means your much more susceptible to interference from external sources, which may then cause other problems. Using highly directional antennas can help, as well as allowing a greater distance to be achieved.

Next, still thinking about only one of your three right-hand side networks, add the wired link in parallel to the wireless link and think about spanning-tree. The first thing to note is that the AP does not participate in spanning-tree at all, but it will "blindly" forward BPDUs, so you can think of the wireless bridge link as if it were just another wire between the switches on the left and right sides of your diagram.

The first problem is that you end up with the AP's interface that is in bridge mode seeing and learning MAC addresses from the whole network. Although the network port that it connects to may be in an STP BLOCKING state, this only affects traffic coming in to that port (i.e. from the AP) - broadcast frames from hosts on the left-hand side of the network will traverse the wired link and still be presented to the Ethernet port on the right-hand side AP and therefore trigger MAC learning. This is more likely to hit the table size limits for the forwarding engine, especially if there are hundreds or thousands of hosts on the right-hand side of the network.

The second problem is that Aerohive have implemented their own mechanism for loop prevention to allow for complex multi-hop mesh designs to be loop-free, which can conflict with what spanning-tree is trying to achieve. This is primarily based on AMRP neighbour discovery packets going between the APs over the wired connection, so you need to have a way to prevent that from happening. This involves removing/disallowing the AP management VLAN from the trunk going up to the right-hand AP. The management VLAN therefore also needs to be a separate VLAN from user VLANs.

Thirdly, there are behaviours built into the mesh functionality which can result in APs going into an "island" mode of operation where basically they stop providing backhaul services and can become unreachable even from a management perspective.

Finally, your diagram shows you trying to support wireless clients on the APs providing the bridge link. The wireless clients connected to the right-hand AP will backhaul to the wired network over the wireless bridge link (and then potentially back over the wired link to the original location!). This is obviously inefficient. Remember that AP's backhaul default route is over the wireless backhaul and also the port on the Ethernet switch will be in a BLOCKING state anyway.

So now, add in your desire to have three separate wireless links terminating on the same AP on the left-hand side of the network. This is even more of a problem. What would happen if just one of the wired links were to fail? In this case, traffic from the right-hand side would start going over the wireless bridge to the left-hand side but with a single-instance STP (e.g. RSTP), the switch port on the left-hand switch is still going to be in a BLOCKING state (otherwise we'd have a loop for the other two right-hand locations) - remember the AP is not participating in spanning-tree itself - you have to think of it acting more like an Ethernet hub. If the VLAN topology allowed it, you may be able to get away with a multi-instance spanning-tree protocol that uses tagged BPDUs (e.g. PVST+), but it really is starting to get very messy indeed.

As you may have gathered, I have looked into this sort of topology in the past, and spent quite some time with Aerohive's design team trying to find a workable solution. In the end, we got to a configuration (for the simple case of a single wired and single wireless link) that worked but it was far from ideal and was not, in my view, sufficiently reliable for a production environment.

I would therefore advise against using Aerohive APs for this kind of topology - there are other vendors that provide solutions specifically designed to provide wireless bridge capabilities and I think you will save yourself a lot of pain using something designed to do this specific job rather than trying to use functionality that was really designed for a different purpose.
Photo of Kamil Słodecki

Kamil Słodecki

  • 4 Posts
  • 0 Reply Likes
Brian - just to clarify - the country code is set to UK but I don't think it matters anymore as Roberto put a period to this topic :)

Roberto - many thanks for making me read that book.. I was actually very informative. I am not going to reply with another book, just quick comments.

1. The goal I was trying to achieve was to add some redundancy with no extra cost - using what I have already. The real topology is obviously bigger and I must admit I was very sceptic designing this diagram but I thought I will give it a go anyway.

2. The minute after I finished the diagram I realised that this may not work as expected due to STP (which you explained very well). Then I thought I may need to enable the STP on all AP's but I don't really want them to participate in the process (I wasn't aware of the AP's limits btw).

3. Aerohive solution is designed to do other things and this is my lesson I learnt here. 

I used in past other vendors that simply do PTP links and it does work very well. Looks like this is the way to go and I was expecting that.
I just wanted someone more experienced to confirm that :) 

Once again, many thanks to both Brian and Roberto for your inputs here. Really appreciate it.

Now the funny part - I have created the "mesh lab" and one of my switches is Dell added to the HiveManager NG. This switch managed to generate over 11k event logs (all STP root election) during a switchover. Unfortunately all the messages appear in HiveManager and I don't see a "remove all" button.
Good night.