Connectivity issues whole network dropping

  • 1
  • Question
  • Updated 2 years ago
Currently working with support to figure out what is going on but I wanted to reach out here to see if anyone has had a similar issue.  I have 2 High schools that keep dropping completely (just the wired network) on PARCC test day.  We have several other schools doing large testing groups with zero issues.  What happens is that the minute the testing begins, the wireless network drops.  The AP's lose CAPWAP connectivity and if a client isn't authenticated and served a DHCP address it's not going to work.  It almost seems malicious to me.  Kind o fl ike a DoS attack on the wireless network.  The wired network stays up and runs fine.  If you look in the DHCP scopes and reconcile the scope we get tons of erroneous MAC addresses from DHCP/BOOTP clients.  We do not serve BOOTP.  From the info i could find online, those mac addresses come from a PXE booting machine.  The entire school (wireless only) goes down in about 3 minutes after this starts.



We are running ap 121's connected to HP 2920 with 10GBE backhauls to our HP 5406.  I am running current firmware on all hardware and the hivemanager is on our WAN running in a VM.  Clients are mostly Dell Chromebooks.

Any help would be greatly appreciated.  Attached is a screen shot of the DHCP server
Photo of Christopher Porter

Christopher Porter

  • 2 Posts
  • 0 Reply Likes

Posted 2 years ago

  • 1
Photo of Gary Babin

Gary Babin

  • 21 Posts
  • 5 Reply Likes
Chris - if this happens at the onset of PARCC testing have you looked at the cacheing server in that building? Maybe it's Ethernet port is haywire?
Photo of Roberto Casula

Roberto Casula, Champ

  • 231 Posts
  • 111 Reply Likes
If you look at the unique IDs, they are an ASCII encoding of the IP address of that lease (31="1", 30="0", 2e="." etc.).

Normally, clients use their MAC address as the unique ID for DHCP requests, but they don't have to.

I would check your DHCP logs (in the %systemroot%\system32\dhcp folder) to see if you can find more info as to what is going on. I would also run a Wireshark capture to look at the broadcast traffic on the network, or if you can run it on the DHCP server itself.

From a quick Google, LOADS of people have had this exact symptom on their DHCP server before. I did find a couple of these which suggested that this exact behaviour was being caused by a bug in Kaspersky Anti-Virus, e.g. here

If that isn't your issue, then it is worth checking the LAN switches to see if the MAC address table is filling up with bogus MAC addresses. There have been issues in the past with certain Intel motherboards where the NIC card, when the machine is asleep but the NIC is still active waiting for WoL magic packets, generates rogue packets with sequentially increasing source MAC addresses that cause switches' MAC address tables to fill up. Perhaps something similar.

Good luck - and let us know how you get on - I like these weird ones!
Photo of rbentley

rbentley

  • 12 Posts
  • 0 Reply Likes
Does this happen on all your DHCP scopes or just the 'Management' one?
Are the computer that are being used for testing on the wired or wireless network?
Photo of Christopher Porter

Christopher Porter

  • 2 Posts
  • 0 Reply Likes
All of the scopes. I took some steps to mitigate, i'll post more here tomorrow. Been all clear since.