IP-based monitoring of remote Access Points with dynamic IP through L2 tunnel - best method?

  • 1
  • Question
  • Updated 3 years ago
  • Answered
We have a scenario where each AP170 is installed behind a cable modem with a public IP via DHCP, and all traffic tunneled via Layer2 tunnel to a central CVG running as L2 VPN gateway.

We need to monitor all Access Points with our Nagios-based monitoring system, which turned out to be a little challenge :-). Each AP receives a public IP address, and it is not possible by the carrier to allow fixed IPs or configure DHCP reservations.

DynDNS updates from each AP could be a solution, but to my knowledge, this is not supported (correct?).

As the L2 VPN gateway assigns a private IP address to each tunnel end-point, we are currently reading out those assigned IPs and use them to access the APs via SNMP:

#show vpn gre-tunnel
Tunnel table:
T=Type; Z=Zone; PN=policy numbers; Age Out=idle time of the tunnel since last receive packet TXs=TX packets; TXE=TX errors; RXs=RX packets; RXE=RX errors; Type: G=General route encapsulation; O=Other tunnel; Zone: A=Access; B=Backhaul; Total entries: 10 ID T Z PN Age Out Src IP Dst IP TXs TXE RXs RXE ---- - - --- -------- --------------- --------------- -------- ---- -------- ---- 11 G B 2 1 10.255.230.15 10.255.230.78 3494 0 526 0 8 G B 2 2 10.255.230.15 10.255.230.74 3066 0 39 0 5 G B 2 8 10.255.230.15 10.255.230.75 3693 0 20 0 10 G B 1 0 10.255.230.15 10.255.230.77 38144 0 26473 0 3 G B 1 4 10.255.230.15 10.255.230.82 21596 0 4921 0 7 G B 6 0 10.255.230.15 10.255.230.71 164280 0 125090 0 9 G B 3 2 10.255.230.15 10.255.230.72 95950 0 26682 0 4 G B 1 0 10.255.230.15 10.255.230.81 939553 0 457775 0 2 G B 22 0 10.255.230.15 10.255.230.70 458237 0 230612 0 1 G B 14 0 10.255.230.15 10.255.230.73 222519 0 42869 0

And then, per AP:

snmpwalk -v 2c -c hivecommunity 10.255.230.74
iso.3.6.1.2.1.1.1.0 = STRING: "HiveAP170, HiveOS 6.1r2 release build1359" 
iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.4.1.26928.1
iso.3.6.1.2.1.1.3.0 = Timeticks: (190217177) 22 days, 0:22:51.77
iso.3.6.1.2.1.1.4.0 = STRING: "admin@aerohive.com" 
iso.3.6.1.2.1.1.5.0 = STRING: "AP-Name" 
iso.3.6.1.2.1.1.6.0 = STRING: "\"Location\"" 
iso.3.6.1.2.1.1.8.0 = Timeticks: (200) 0:00:02.00
iso.3.6.1.2.1.1.9.1.2.1 = OID: iso.3.6.1.6.3.1
...


This works, but is more a workaround, as the IP addresses could be re-assigned to different APs, e.g. when the CVG reboots.

Questions:

- Is there a way to fix tunnel endpoint IP addresses from the L2VPN gateway?
- Or can I add an additional management IP to each AP for monitoring purpose?
- Or is there a way to send DynDNS updates from each AP?
- Any better idea to realize this kind of monitoring?


Thanks for any input :-)

carsten


Photo of Carsten Buchenau

Carsten Buchenau, Champ

  • 356 Posts
  • 117 Reply Likes

Posted 4 years ago

  • 1
Photo of Gregor Vucajnk

Gregor Vucajnk, Official Rep

  • 74 Posts
  • 27 Reply Likes
Hi Carsten, 

You've pretty much nailed the workaround. However, consult the new 6.1r5 release document to see if you can utlise the new RESTtful APIs: http://www.aerohive.com/330000/docs/help/english/6.1r3/hm/api/

I have to play with those myself so I will be more helpful as I learn about the implementation.

Gregor
Photo of Simon Hogg

Simon Hogg

  • 13 Posts
  • 8 Reply Likes
Hi Gregor,

Just to see if you ever had time to play with this JSON API?  We are now using GRE tunnels only in this example and therefore we don't have the IPSEC endpoints to use for our monitoring...so being able to get AP info from the HM either via SNMP or JSON would be really cool.

Do you know if anyone has created projects with this API, or has some form of framework for us to start from?

Cheers,

Simon
Photo of Simon Hogg

Simon Hogg

  • 13 Posts
  • 8 Reply Likes
Ok...we found a way to do this now...at least a starting point.  First of all you need to enable API access in the hivemanager settings, and set a user and password there.

Then...the API is open to be queried.  We wrote a very simple PHP script just to test this works.  It simply creates a connection and downloads the JSON file which is then parsed to see if any of the APs listed as not connected, and then print their names.

The username and password need to be passed using basic authentication, so this needs to be a base64 conversion of username:password (can be done online: http://www.base64encode.org/)....probably other ways too but this worked for us.

The test script we created is below in case it is of interest to anyone:


<?php

// set HTTP header and enter auth details
$headers = array(
'Content-Type: application/json', 'authorization: Basic xxxxxxxxxxxxxxx');

$url = 'https://hivemanager.xxxx.yyy/hm/api/v1/devices';

// Open connection
$ch = curl_init();

// Set the url, number of GET vars, GET data
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, false);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

// Execute request
$result = curl_exec($ch);

// Close connection
curl_close($ch);

// get the result and parse to JSON
$result_arr = json_decode($result);


// Some tests to see the full data. Uncomment to test
//print_r($result_arr);
//var_dump($result_arr);

// Iterate through the result and print any APs that are down
foreach($result_arr as $ap)
{
if($ap->connection == "0")
{
echo $ap->hostName."\n";
}
}

?>

(Edited)
Photo of Gregor Vucajnk

Gregor Vucajnk, Official Rep

  • 74 Posts
  • 27 Reply Likes
Hi Simon. Alas I still haven't invested my time in this so thank you for posting your findings. If I manage to figure out something else I'll post it here.

Gregor
Photo of J. Goodnough

J. Goodnough, Champ

  • 266 Posts
  • 32 Reply Likes
Carsten, what sort of information do you monitor with your Nagios implementation?
Photo of J. Goodnough

J. Goodnough, Champ

  • 266 Posts
  • 32 Reply Likes
Nice - I also use check_mk - I'll have to consider setting this up.
Photo of Simon Hogg

Simon Hogg

  • 13 Posts
  • 8 Reply Likes
Really love CMK...so nice to use.  For the active connections, we wrote a custom plugin for that.  At some point we should upload back into the community, but if it helps you, please see below:

#!/usr/bin/python
#This is a quick SNMP check for AeroHive Acess Points.
#
#OID=.1.3.6.1.4.1.26928.1.1.1.2.1.2.1.3   returns an array of strings (client names) or fails if no clients connected
#
#
def inventory_aerohive_clients(info):
        return [ (None, None) ]


def check_aerohive_clients(item, params, info):
        sessions = len(info)
        perfdata = [("Active Sessions", sessions)]
        return (0, "OK - Currently {0} Active Sessions.".format(sessions), perfdata)

snmp_info['aerohive_clients'] = ( ".1.3.6.1.4.1.26928.1.1.1.2.1.2.1", ["3"] )

check_info['aerohive_clients'] = (check_aerohive_clients, "Active Sessions", 1, inventory_aerohive_clients)

snmp_scan_functions['aerohive_clients'] = lambda oid: ".1.3.6.1.4.1.26928" in oid(".1.3.6.1.2.1.1.2.0")
Photo of J. Goodnough

J. Goodnough, Champ

  • 266 Posts
  • 32 Reply Likes
That looks amazingly helpful. I'm not sure when I'll get to working on this, but I'll definitely report back when I do.
Photo of Stephan Hughson

Stephan Hughson

  • 8 Posts
  • 2 Reply Likes
Hi, Simon, thanks for the code above. I've copied and extended it a bit.
I'll share it here in case it's handy for anyone else using Nagios.



<?php

$auth = $argv[1];
// base64 encoded username:password for the Aerohive API
$url = $argv[2];
// e.g. https://hivemanager.xx.yy/hm/api/v1/devices

// set HTTP header and enter auth details
$headers = array(
    'Content-Type: application/json', 'authorization: Basic ' . $auth);

// Open connection
$ch = curl_init();

// Set the url, number of GET vars, GET data
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, false);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

// Execute request
$result = curl_exec($ch);

$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
  $curl_errno = curl_errno($ch);
  if ($http_status!=200) {
    echo "Cannot get information from the API. Curl returned error number " .$curl_errno;
exit (3);
}

// Close connection
curl_close($ch);

// get the result and parse to JSON
$result_arr = json_decode($result);

// Some tests to see the full data.  Uncomment to test
//print_r($result_arr);
//var_dump($result_arr);

// Iterate through the result and print any APs that are critical.

$critical_list = array();
$warning_list = array();
$down_list = array();

foreach($result_arr as $ap)
{

    if($ap->connection == "0")
    {
        $down_list[] = ($ap->hostName);
    }

    if($ap->alarm == "Critical")
    {
        $critical_list[] = ($ap->hostName);
    }

    if(($ap->alarm == "Major") || ($ap->alarm == "Minor"))
    {
        $warning_list[] = ($ap->hostName);
    }

}

echo "AP status: " . sizeof($warning_list) . " warning, " . sizeof($critical_list) . " critical, " . sizeof($down_list) . " down.";

if (sizeof($critical_list)  > 0) { $output="\n\nCritical: ";    foreach($critical_list as $child)    $output .=  $child . ", "; echo rtrim($output, ", "); }
if (sizeof($warning_list)   > 0) { $output="\n\nWarning: ";    foreach($warning_list as $child)    $output .=  $child . ", "; echo rtrim($output, ", "); }
if (sizeof($down_list)      > 0) { $output="\n\nDown: ";    foreach($down_list as $child)        $output .=  $child . ", "; echo rtrim($output, ", "); }

if ((sizeof($warning_list) == 0) && (sizeof($critical_list) == 0) && (sizeof($down_list) == 0)) exit (0); 
if ((sizeof($warning_list)  > 0) && (sizeof($critical_list) == 0) && (sizeof($down_list) == 0)) { exit (1); } else { exit (2); }

?>




To run it, I do:

php ./hivemanager_api.php "bmF-changed...-mZw==" "https://hivemanagffer.xx.yy.zz/hm/api/v1/devices";



# 'hivemanager_api' command definition
define command{
    command_name    hivemanager_api
    command_line    /usr/bin/php /usr/lib/nagios/plugins/hivemanager_api.php $ARG1$ $ARG2$
    }



define service{
    use            generic-service
    host            my.aerohive.server.address
    service_description    Aerohive AP Health
    check_command        hivemanager_api!"bmFn--changed...--Z2RmZw=="!"https://hivemanager.xx.yy.zz/hm/api/v1/devices";";
    }




Output looks like:

AP status: 0 warning, 0 critical, 1 down.
Down: the-aps-affected-separated-by-commas

It will tell you about Major, Minor and Critical APs as well.


I hope this helps someone. Sorry for any bugs :-).



keywords: nagios aerohive api monitor hivemanager api
Photo of Simon Hogg

Simon Hogg

  • 13 Posts
  • 8 Reply Likes
Looks good Stephan, thanks for sharing!!
Photo of Stephan Hughson

Stephan Hughson

  • 8 Posts
  • 2 Reply Likes
Here's a Nagios plugin for clients per AP if it helps anyone. Not fully tested yet in anger but seems to be working so far. Please let me know if you find bugs.


<?php

$auth = $argv[1];
// base64 encoded username:password for the Aerohive API
$url = $argv[2];
// e.g. https://hivemanager.CHANGEDchangedCHANGED/hm/api/v1/devices

// set HTTP header and enter auth details
$headers = array(
    'Content-Type: application/json', 'authorization: Basic ' . $auth);

// Open connection
$ch = curl_init();

// Set the url, number of GET vars, GET data
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, false);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

// Execute request
$result = curl_exec($ch);

$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
  $curl_errno = curl_errno($ch);
  if ($http_status!=200) {
    echo "Cannot get information from the API. Curl returned error number " .$curl_errno;
exit (3);
}

// Close connection
curl_close($ch);

// get the result and parse to JSON
$result_arr = json_decode($result);

// Some tests to see the full data.  Uncomment to test
//print_r($result_arr);
//var_dump($result_arr);

// Iterate through the result and print any APs that are critical.

$critical_list = array();
$warning_list = array();
$ok_list = array();

foreach($result_arr as $ap)
{

$clients_for_this_device = $ap->clients;

    if($clients_for_this_device < 21)
    {
        $ok_list[] = ($ap->hostName);
    }

    if(($clients_for_this_device > 20 ) && ($clients_for_this_device < 30))
    {
        $warning_list[] = ($ap->hostName);
    }

    if($clients_for_this_device > 30)
    {
        $critical_list[] = ($ap->hostName);
    }

}

echo sizeof($ok_list) . " APs with 0-20 users, " . sizeof($warning_list) . " with 20-30 users, " . sizeof($critical_list) . " with over 30 users.";

if (sizeof($warning_list)   > 0) { $output="\n\nAPs with 20-30 users: ";    foreach($warning_list as $child)    $output .=  $child . ", "; echo rtrim($output, ", "); }
if (sizeof($critical_list)  > 0) { $output="\n\nAPs with over 30 users: ";    foreach($critical_list as $child)    $output .=  $child . ", "; echo rtrim($output, ", "); }

if ((sizeof($warning_list) > 0) && (sizeof($critical_list) == 0)) exit (1); 
if (sizeof($critical_list) > 0) exit (2);
if ((sizeof($warning_list) == 0) && (sizeof($critical_list) == 0)) exit (0); 

?>