AP230s not passing traffic occassionally

  • 2
  • Question
  • Updated 3 years ago
Customer has informed me that some clients, iPads, chromebooks and windows PCs alike, aren't able to obtain ip addresses periodically and the only fix is to reboot the AP.  Everything else works as far as the network policy, vlans and dot1x goes as other users in the same building using the same network profile can connect no problem. Its also not a dhcp scope or lease issue as other users in the same building are able to grab new ip addresses.  It seems as though the AP goes into a stuck state handing out 169 addresses until it is rebooted.  It is up in HM as connected while the customer is experiencing the issue as well.  District has about 1800+ AP230s (1x1 deployment) and they have told me rebooting 10-15 APs a week has become the norm.  THe problem is wide spread across 33 buildings all using different network profiles.  I'm waiting for the problem to happen again before I can run a client monitor session on a client trying to connect to a problematic AP to see where the client fails.  In the meantime, has anyone seen this issue before? 
Photo of Josh Grzelakowski

Josh Grzelakowski

  • 11 Posts
  • 0 Reply Likes

Posted 3 years ago

  • 2
Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
What HiveOS is being used?

Under your Network Policy and Additional Settings and QoS Settings, do you have the tickbox Dynamic Airtime Scheduling selected?  

If so, try turning it off and see if that issue goes away.  I've seen similar symptoms before and disabling DAS seemingly fixed it.  
Photo of Josh Grzelakowski

Josh Grzelakowski

  • 11 Posts
  • 0 Reply Likes
6.4r1e.2116

Yes, dynamic airtime scheduling is on per the high density deployment guide.  If I uncheck this box and update the APs at a single school, will this affect client connectivity, both currently connected and trying to connect?  In other words, could this be done with the clients being affected?
Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
I cannot remember if clients will get booted briefly with this change, but what I did at one site to see if it helped was as follows:

Once I got notified of an AP that "quit passing traffic"
I SSH'd into the AP and issued a "no qos airtime enable" (this disables DAS on the particular AP).
After doing this, clients in the area that were seemingly connected to this AP started working immediately.  

As a real down n dirty test while I was doing this, I had one of the folks onsite with an iOS device in the area and was attempting to ping his device.  I could almost like clockwork ping/not ping his device by enabling and disabling DAS (via the above CLI command).

Your mileage may vary as you could be seeing something differently than what I had, but the symptoms sound familiar...
Photo of Josh Grzelakowski

Josh Grzelakowski

  • 11 Posts
  • 0 Reply Likes
I will definitely try that when this issue happens again.  What exactly does that feature do?
Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
Below is the details from the help file...

Dynamic Airtime Scheduling

Traditional QoS methods classify client traffic into queues based on traffic type and then schedule the transmission of the traffic based on data rates from the queues (128 Kbps, for example). In a WLAN, this means that a slower client (802.11b, for example) uses significantly more airtime to transmit the same amount of data as a much faster client (802.11n). To address this and make airtime access more equitable, Aerohive provides airtime-based QoS. Instead of just using bandwidth in the QoS calculations, APs allocate airtime per client, per traffic class, and per user profile by dynamically calculating airtime consumption per packet. In the case where multiple types of clients (802.11a, b, g, or n) are active in the same WLAN, all clients receive the same amount of airtime, such as 10 ms for example, regardless of the type of client. The 11n client might be able to send 128 Kbps of traffic in the allocated slot, while the 11b client might only send 50 Kbps, but both clients get the same amount of airtime to use however they can.

Here is a visual representation of the difference between bandwidth-based scheduling and airtime-based scheduling when two clients are transmitting at different data rates.

Dashes ( — ) indicate airtime for frames transmitted at a low data rate.

Bullets ( • ) indicate airtime for frames transmitted at a high data rate.

Bandwidth-based scheduling:

— • — • • — • — — • — • — • — •

Airtime-based scheduling:

— • • • • — • • • • — — — — — —


The faster client might be using 802.11n and the slower client 802.11a/b/g, or they might both be using the same protocol but one is farther away and must use a slower speed than the other.

As you can see, with bandwidth-based scheduling, both the slow and fast clients finish at the same time regardless of their data rates, and they compete the entire time for the air. Because both clients have an equal opportunity to transmit frames, the faster client's throughput slows down to the rate of the slower client.

On the other hand, with Dynamic Airtime Scheduling, both clients get their proportion of airtime. The fast client finishes four times faster, and the slower client finishes at the same time as it did with bandwidth-based scheduling. The fast client is rewarded, and the slow client is not penalized.


To combat this, we typically just disable 802.11b auth attempts and/or set a non 802.11b data rate as a base data rate.

Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
Additional details that I missed...

The following is a summary of some benefits of Dynamic Airtime Scheduling:

Multiple-service WLAN infrastructure: When integrated with service-based QoS scheduling in user policies, Dynamic Airtime Scheduling lets you manage the air optimally to support different application types.

Dense deployments: When there are many clients and the air is congested with wireless traffic, Dynamic Airtime Scheduling ensures that faster clients get off the air faster reducing the likelihood of collisions and contention among all client traffic.

Sparse or Partial Deployments: When a few clients are spread out at various distances from the AP, Dynamic Airtime Scheduling prevents fringe or distant clients from slowing down closer, faster clients and allows for phased roll-outs.

Environments with a mixture of 802.11 a/b/g/n clients: When there is a mixed environment with clients using different protocols when connecting to the AP, Dynamic Airtime Scheduling enables the faster clients to get the benefit of their speed without penalizing the legacy clients.


To enable dynamic airtime scheduling, select the check box. To use the traditional, bandwidth-based method, clear it. By default, dynamic airtime scheduling is disabled.

  • After you enable airtime scheduling, an AP applies it to traffic assigned to admin-defined user profiles but not to traffic assigned to the predefined user profile "default-profile". Therefore, if you want to apply airtime-based QoS to client traffic, make sure that the SSIDs reference only user profiles that you or another admin created.
Photo of Josh Grzelakowski

Josh Grzelakowski

  • 11 Posts
  • 0 Reply Likes
Thanks for providing that.  I will keep you posted.
Photo of Dawn Douglass

Dawn Douglass

  • 67 Posts
  • 3 Reply Likes
Could it be that the AP acting as a DHCP server is overwhelmed providing other services and therefore is unable to hand out IPs on an intermittent basis so the clients assign themselves a 169.x IP?
Photo of Josh Grzelakowski

Josh Grzelakowski

  • 11 Posts
  • 0 Reply Likes
This issue happened twice yesterday and running the no qos airtime enable command on the AP CLI fixed the problem instantly.

A few questions I have though.

Why is this supposed to be enabled per the high density deployment guide?
Why is this setting causing the AP to not send ANY traffic to clients?
If I disable it globally, how will that affect the environment?  Lower speeds or anything like that?  Will end users see something over time?  Or is it just a setting that tries to help with prioritizing client traffic?

Any additional information would be extremely beneficial.
Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
Josh,

Glad that may have been your fix.

I wish I could provide additional information in regards to this but I cannot. I do not believe the issue to exist in any APs except the AP230 as like you, I've always enabled DAS. The issue exist d in several HiveOSs as well. I have not tried turning it back on with the latest 6.6rx codes.

If I ever have time to delve further into the issue, I'll be sure to update, but when the problem presented itself to me, I (and the end user) was more worried about resolving the issue than spending time and resources (and end user frustration) working with Aerohive support to try to find out more details.
Photo of Roberto Casula

Roberto Casula, Champ

  • 231 Posts
  • 111 Reply Likes
There have been a number of issues related to DAS in the past, including high CPU utilisation and connectivity issues. I've also seen this behaviour at some customers on the 6.4 and older releases, but at other customers with the same software and similar configuration, I don't see the issue. And it's very intermittent and can't be reproduced on demand so troubleshooting it is not easy because the impetus is just to get things working again. I've not really had an opportunity to troubleshoot or raise with Aerohive.

I haven't seen the issue so far with 6.6 code, but the main two customers that have experienced this most frequently haven't yet gone to 6.6, so I wouldn't like to definitively say this problem doesn't exist in 6.6. Likelihood is we will put them on the 6.5 golden release when it comes out as stability is the main requirement, not new features.

DAS was a great feature in the early days of .11n where there were a significant number of legacy clients also present. There is a bit of a question mark over how much benefit there is in using DAS if all your clients are .11n/ac. Unless you are also using SLAs in the user profile to try to give some priority access to certain groups of users, you may find that the benefits of DAS are so small as to not be worth the extra software complexity that results.
Photo of Josh Grzelakowski

Josh Grzelakowski

  • 11 Posts
  • 0 Reply Likes
Disabling dynamic airtime scheduling has fixed the issue.  The customer hasn't reported the issue in over 2 weeks.  Thanks everyone for their help and assistance.
Photo of Brian Powers

Brian Powers, Champ

  • 396 Posts
  • 92 Reply Likes
Glad it fixed things!
Photo of Simon Hogg

Simon Hogg

  • 13 Posts
  • 8 Reply Likes
I can confirm we saw the same symptoms with AP230's as well, and they were resolved by the removal of DAS it seems.

Thanks to the forum for the great troubleshooting.