AP's require reboot before update

  • 3
  • Question
  • Updated 4 years ago
  • Answered

Is anyone else having issues where they need to reboot an AP before config changes take? They seem to work fine with the uptime, but if the uptime is 25+ days, 95% of the time I upload a config, it fails and requires me to schedule downtime to do my work. Uptime is supposed to be a good thing, but it's killing me.


Photo of Jeremy Stewart

Jeremy Stewart

  • 47 Posts
  • 0 Reply Likes

Posted 4 years ago

  • 3
Photo of Nick Lowe

Nick Lowe, Official Rep

  • 2491 Posts
  • 451 Reply Likes
What version of HiveOS is this with? Can you reproduce it with 6.2r1?

I have personally experienced a similar issue when performing HiveOS updates but not configuration updates on HiveAP 330s.
(Edited)
Photo of Jade Rampulla

Jade Rampulla

  • 36 Posts
  • 1 Reply Like
I've upgraded a virtual appliance and 2 HMOL accounts from 6.1r6 to 6.2r1 and didn't have this problem. I tried upgrading a 3rd HMOL account today and ran into the exact same problem. The other upgrades had AP121's, AP330's, AP230's, AP370's, and BR's. The one I had problems with had all AP121's.

I checked these options:




I got this error on most of them:

Photo of BJ

BJ, Champ

  • 374 Posts
  • 45 Reply Likes
I seem to recall reading that AP121s required reboots prior to updating to 6.2. I'll try to find that documentation.

Best,
BJ
Photo of Andrew Garcia

Andrew Garcia, Official Rep

  • 368 Posts
  • 120 Reply Likes
The AP121 does not have a backup firmware image partition.  APs that do not have a backup firmware partition require more memory than usual be available to reliably save and install a firmware update. We put in logic to check memory and require a reboot if memory levels are insufficient, which is likely why you are seeing APs with extended uptime require the reboot first.
Photo of Jeremy Stewart

Jeremy Stewart

  • 47 Posts
  • 0 Reply Likes

Ok.. Firmware upgrade aside, does the same apply for config changes that shouldn't require a reboot at all? I can get past the FW upgrade, because i'd need to schedule downtime anyhow (still an issue btw).

If anyone got an email on the original post to this, you know where i'm going.

(Edited)
Photo of Andrew Garcia

Andrew Garcia, Official Rep

  • 368 Posts
  • 120 Reply Likes
It looks like there was a bug in HiveOS 6.1r6 (and 6a) that led to this need for a reboot before a config update. It is fixed in HiveOS 6.2r1.
(Edited)
Photo of Jeremy Stewart

Jeremy Stewart

  • 47 Posts
  • 0 Reply Likes
When are we expecting an update to HMOL?
Photo of Nick Lowe

Nick Lowe, Official Rep

  • 2491 Posts
  • 451 Reply Likes
If your HMOL has not yet been moved to 6.2r1a and you are not being shown a 'flying/flashing bee' to update yourself, you should ask your support provider to expedite things.
Photo of twisted

twisted

  • 1 Post
  • 0 Reply Likes
Yes.  I had this happen today with 5 230's out of 25.   I got the same error with the SCP username /pass invalid.  I had to reboot them to take the updates.  It seems to be around 30 days, since last reboot.   I use HMOL.   6.1r6a
(Edited)
Photo of Jeremy Stewart

Jeremy Stewart

  • 47 Posts
  • 0 Reply Likes

I get "unknown errors" with CLI commands referencing security objects. Mine are all 121's. I have had issues where they would need to be rebooted before upgrading too. It's a bigger concern that delta changes are not taking, I shouldn't have to schedule downtime to update an SSID nobody else is connected to.

Running HMOL on 6.1r6a

(Edited)
Photo of Steven Bateman

Steven Bateman

  • 65 Posts
  • 12 Reply Likes
I've also had the issue on many of the different HMOL instances that I manage, using models ranging from AP121 and AP330/350, to the AP230 and BR200-WP. All of them are running 6.1r6a.1794, and most of them were either already on that version or running 6.1r6.1779. The majority of them were longer uptimes (>30 days)

My symptoms will normally be:

  • Unable to complete a delta upload
  • Configuration audit shows entire config needing to be applied, which tells me that the AP is not responding to the "show config run" request with the correct response
  • Will also get the SCP username/pw error on delta uploads and sometimes complete uploads
  • "Show Log" will return no data
  • "Show Running Config" will return no data
  • Tech logs fail to generate correctly
  • AP refuses to set up SSH connection to HM.
  • "Show CPU" and "Show Memory" commands return data and it looks normal.

The workaround is to reboot the AP or BR.

Sometimes, there are specific entries in the flash logs that a memory overflow occurred:

2014-10-05 06:02:55 err     ah_top: process php-cgi is killed by PM because memory is overflown:10244--->10240: No such file or directory
2014-10-05 01:28:29 err     ah_top: process php-cgi is killed by PM because memory is overflown:10248--->10240: No such file or directory

Other times, there are weird error messages from interfaces, even if I've specified them as Admin State: Down:

2014-10-06 13:55:08 err     last message repeated 11 times
2014-10-06 13:54:13 err     last message repeated 7 times
2014-10-06 13:53:38 err     ah_brd: ah_brd_port_report_get_port_content, get port: eth0 IP address failed.: Cannot assign requested address

I've piloted 6.2r1 on one HM so far, but as it was released recently and required reboots anyway so I doubt I'll see useful data for another few weeks.