Suspicious egress udp 3000 traffic

  • 1
  • Question
  • Updated 5 years ago
  • Doesn't Need an Answer
Hello all,
I'm reaching out to the forum in hopes someone else has experienced this and there is a solution. Our organization has hundreds of Aerohive AP's installed. I recently discovered in my firewall logs that all of these AP's are sourcing traffic over udp-3000 to very suspicious destinations. IP's that terminate in China, Malaysia, japan, etc. Aerohive techicians said this is not expected behavior of the devices so I blocked the traffic at my perimeter but that hasn't stopped the devices from sourcing the traffic. Does anyone know why these AP's would be doing this? And it's not random... all the AP's in my building... at least 65 of them all start trying to talk to the same destination IP's. For example, three unique destination IP's will all be tried repeatedly for hours by each one of my AP's. None of my LAN guys can determine whats generating the traffic.
Does anyone else know of this behavior??
Thanks!

Mike
Photo of Mike

Mike

  • 2 Posts
  • 0 Reply Likes

Posted 5 years ago

  • 1
Photo of Andrew MacTaggart

Andrew MacTaggart, Champ

  • 483 Posts
  • 86 Reply Likes
Do you have captures?

when you say the traffic originates from the AP, I assume that the source is the AP management interface. This assumes that you have user data traffic separated from AP management traffic.

Can I am assume your setup is using HMOL and requires the APs to phone home.

Have you looked up the suspicous destination addresses in dshield.org or some other service?
maybe list them here

What version of code and HM are you running?

I just ran a packet capture and I actuall see simular traffic, but it is running from source AP mgmt IP to another AP mgmt IP.

the source port is listed as 3001 - wireshark says redwood-broker and destination port 3000 hbci, but lists the protocol is DIS - Distributed Interactive Simulation

I am also seeing src port 3002 exlm-agent dst port 3000 wireshark says the protocol is DIS - Distributed Interactive Simulation.

I can only assume wireshark is not understanding the protocol

I would suspect that these are the areohive protocols coordinating the APs.

maybe

AMRP (Aerohive Mobility Routing Protocol) – Provides HiveAPs with the ability to perform automatic neighbor discovery, MAC-layer best-path forwarding through a wireless mesh, dynamic and stateful rerouting of traffic in the event of a failure, and predictive identity information and key distribution to neighboring HiveAPs. This provides clients with fast/secure roaming capabilities between HiveAPs while maintaining their authentication state, encryption keys, firewall sessions, and QoS enforcement settings.
ACSP (Aerohive Channel Selection Protocol) – Used by HiveAPs to analyze the RF environment on each channel within a regulatory domain and to work in conjunction with each other to determine the best channel and power settings for wireless access and mesh. ACSP minimizes co-channel and adjacent channel interference in order to provide optimized application performance.
DNXP (Dynamic Network Extension Protocol) – Dynamically creates tunnels on an as-needed basis between HiveAPs in different subnets, giving clients the ability to seamlessly roam between subnets while preserving their IP address settings, authentication state, encryption keys, firewall sessions, and QoS enforcement settings. Note that tunnels are not required for clients roaming among APs in the same subnet.

Maybe Aerohive could provide more insight and how to configure wireshark to register these protocols as Aerohive protocols.

3000 HBCI.
RemoteWare Client (unassigned but in widespread use).
3001 Redwood Broker.
3002 EXLM Agent.
RemoteWare Server (unassigned but in widespread use).

Now why you are seeing them going to your firewall with strange destination addresses, will require you to look at your setup.

For me I have an onsight hive and my AP mgmt segment is separate and not routable to the internet.

You may have several locations with HMOL. You will need to dig deep into the destination addresses.

Cheers
A
Photo of Andrew MacTaggart

Andrew MacTaggart, Champ

  • 483 Posts
  • 86 Reply Likes
I would actually love to know the in's and out's of the Aerohive protocols.
So I know what I am looking at.
So I can customize wireshark
So I can figure out why my cisco switches lists them as unknown protocols - though I suspect since they are encrypted this is the reason.
but at least knowning the ports used by each protocol would be helpful.

Cheers
A
Photo of Crowdie

Crowdie, Champ

  • 972 Posts
  • 272 Reply Likes
Port 3000 is the default hive traffic control port.

Hive communications operate at Layers 2 and 3. The default port number for Layer 3 hive communications and for roaming-related control traffic is UDP 3000. If a different service on your network is already using port 3000, you can change the port number that hive members use when exchanging control data. This port number can be any number from 1024 to 65535, as long as the new setting is at least 5 digits greater or less than the current setting. For example, if the current port number is 3000, you can set a new port number higher than 3005.

To test this I would create a new hive with the hive traffic control port set to 4000. Create a new network policy that is assigned to the new hive (this is done in the "Additional Settings" area. Lastly assign the new network policy to a single access point, do a complete upload to the access point and monitor traffic from that access point.

I can't imagine that Aerohive would ever reveal how the co-operative control protocols work at a low level as this would enable SOHO wireless vendors to reverse engineer them or, at least, make their products look like they have.
Photo of Mike

Mike

  • 2 Posts
  • 0 Reply Likes
Hi guys,

Thanks for the replies. Yes, we have done some wireshark trace files of the AP’s but they don’t show the traffic in question.
Our discussions with Aerohive led to the belief that yes, AP’s use this protocol to discover neighboring AP’s but the expectation is that it will never leave our local network.
As for the specifics of the hive configurations, you’ll have to excuse me, but I’m in the security office, not the Lan Operations group so I have no idea how they have these things setup. I’m just monitoring my perimeter and my firewall is logging the traffic.
Here is a screenshot of my firewall log. This is just an 18 second time slice. The source column has the first two octets of my subnet omitted but you get the idea. There are many other destinations that follow the same pattern.

Those destinations resolve as:

37.136.52.0 = 37-136-52-0.nat.bb.dnainternet.fi. Geo information leads to Finland
60.76.139.239 = softbank060076139239.bbtec.net - Geo information leads to Japan

So why my AP’s are trying to go to these destinations is beyond me. We’re located in Washington state.
Where should I tell these guys to look to try to stop this communication?

Thanks again!

Mike
Photo of Andrew MacTaggart

Andrew MacTaggart, Champ

  • 483 Posts
  • 86 Reply Likes
I would suspect that these are co lo's for the HMOL, but only Aerohive could confirm that.