Custom component: EdgeOS

alex3m3ti8 · October 13, 2022, 1:21am

From edgeos.ac48cd8ddea8899b9c33218f0b714313.debug.api.json with a little leading up to openvpn section

          "eth5": {
            "disable": null,
            "duplex": "auto",
            "speed": "auto"
          },
          "eth6": {
            "disable": null,
            "duplex": "auto",
            "speed": "auto"
          },
          "eth7": {
            "disable": null,
            "duplex": "auto",
            "speed": "auto"
          }
        },
        "loopback": {
          "lo": null
        },
        "openvpn": {
          "vtun0": {
            "config-file": "/config/openvpn/us8401.nordvpn.com.udp.ovpn",
            "description": "OpenVPN VPN tunnel",
            "firewall": {
              "out": {
                "name": "LAN_OUT"
              }
            }
          }
        }
      },

from edgeos.ac48cd8ddea8899b9c33218f0b714313.debug.ws.json when the VTUN0 interface is truly up (Enabled and Status Connected)

      },
      "vtun0": {
        "addresses": [
          "10.8.2.9/24"
        ],
        "l1up": "true",
        "multicast": 0.0,
        "rx_bps": 23398.0,
        "rx_bytes": 220703671.0,
        "rx_dropped": 0.0,
        "rx_errors": 0.0,
        "rx_packets": 275133.0,
        "tx_bps": 82049.0,
        "tx_bytes": 81622183.0,
        "tx_dropped": 0.0,
        "tx_errors": 0.0,
        "tx_packets": 191121.0,
        "up": "true"
      }

And after toggling with switch, confirming that the Interface is now Disabled/Status Connected.

      },
      "vtun0": {
        "addresses": [
          "10.8.2.9/24"
        ],
        "l1up": "true",
        "multicast": 0.0,
        "rx_bps": 3965.0,
        "rx_bytes": 236150528.0,
        "rx_dropped": 0.0,
        "rx_errors": 0.0,
        "rx_packets": 299839.0,
        "tx_bps": 75768.0,
        "tx_bytes": 92378624.0,
        "tx_dropped": 0.0,
        "tx_errors": 0.0,
        "tx_packets": 212152.0,
        "up": "true"

Which matches what I mentioned earlier, it appears that EdgeOS is reporting the interface as still up:true even if the configuration is disabled and the interface is actually down.

Other interfaces that are down, like for example ETH6, show up:false

   },
      "eth6": {
        "duplex": "half",
        "l1up": "false",
        "mac": "b4:fb:e4:8d:f8:0b",
        "multicast": 0.0,
        "rx_bps": 0.0,
        "rx_bytes": 0.0,
        "rx_dropped": 0.0,
        "rx_errors": 0.0,
        "rx_packets": 0.0,
        "speed": "10",
        "tx_bps": 0.0,
        "tx_bytes": 0.0,
        "tx_dropped": 0.0,
        "tx_errors": 0.0,
        "tx_packets": 0.0,
        "up": "false"
      },

alex3m3ti8 · October 13, 2022, 1:29am

Also and not a big deal, but I tried to use Developer Tools - Services - Ubiquiti EdgeOS Routers: Update configuration

To enable store debug data.

But that didn’t work. I am able to change Unit to KBytes and back to MBytes and that work and updates edgeos.ac48cd8ddea8899b9c33218f0b714313.config.json. So I had to use SSH and NANO to update the file directly. No big deal, but I think something might be up with that Service.

iDontWantAUsername · October 13, 2022, 1:50am

I have tried editing the file but I am not sure what the new key names are, I tried the following but it didn’t seem to work:

    "unit": "MBytes",
    "log-incoming-messages": false,
    "store-debug-data": false,
    "consider-away-interval": 180,
    "update-api-interval": 60,
    "update-entities-interval": 60

I should also note I found this key had 180 which is my consider-away-interval value: "log-incoming-messages": 180

bar · October 13, 2022, 5:38am

Fixed configuration saving, it reloads the configuration right afterward and working right now

will be avialable in v2.0.6

bar · October 13, 2022, 5:49am

v2.0.6 released with fix to configuration load, save and import processes

thanks again for raising these issues!

iDontWantAUsername · October 14, 2022, 2:56am

Thanks for all your work on this. The latest version (2.0.11) is working well and the update interval has fixed the performance issues I was seeing with updates every second.

One very minor thing I have just noticed is that Last Restart potentially has the wrong data type. It is showing as a graph in Home Assistant with a value of the date as a number (2022.0824200000). The only other date sensor I can find is phone last reboot and this just stores a nicely format date e.g. 2 October 2022 at 15:23.

bar · October 14, 2022, 5:00am

Thanks!

I will check how it was implemented in other components and update here about potential fix

chjohans · October 15, 2022, 2:31pm

I am unable to set both the “Consider away interval”, “Update API interval” and “Update Entities interval” through Developer tools → Services → edgeos.update_configuration. If I enable any of these I just get an error message:
Failed to call service edgeos.update_configuration.extra keys not allowed …

Please see the attached screenshot.

I’m on version 2022.10.1 of HA and version 2.0.11 of this otherwise excellent integration.

bar · October 15, 2022, 3:55pm

Can you please post log with debug level?

bar · October 20, 2022, 12:14pm

v2.0.14

Debugging became easier (less IO and Disk Space)

Removed Store Debug Data switch (Moved to the API endpoints below)
Removed WebSocket messages sensors (Moved to the API endpoints below)
Add endpoints to expose the data was previously stored to files and the messages counters

Endpoint Name	Method	Description
/api/edgeos/list	GET	List all the endpoints available (supporting multiple integrations), available once for integration
/api/edgeos/{ENTRY_ID}/ha	GET	JSON of all HA processed data before sent to entities including messages counters, per integration
/api/edgeos/{ENTRY_ID}/api	GET	JSON of all raw data from the EdgeOS API, per integration
/api/edgeos/{ENTRY_ID}/ws	GET	JSON of all raw data from the EdgeOS WebSocket, per integration

Authentication: Requires long-living token from HA

dm9991966 · October 27, 2022, 10:37pm

Just updated to v2.0.14. With the previous v1.x, the integration was able to see both the main switch0 for the router (Edgerouter 10X) AND the VLAN switches that are defined for my Guest and IoT networks (swtich0.10, switch0.20).

After the update, only the main switch switch0 is showing as discovered device. Would it be possible to get the VLAN switch detectable as well?

monaco · October 28, 2022, 6:31am

Is it possible with this integration to “reboot” PoE access points? Maybe in a way to disable for a few seconds ETH port with PoE (but PoE should be also cut off)

bar · October 28, 2022, 7:28am

Don’t have PoE in my device, don’t know if that’s supported,
Can you please provide API example of the details before and after switching on / off + API call to switch it on / off?

Thanks

bar · October 28, 2022, 12:22pm

2.0.15

Fix unhandled WS session disconnection
Add support for special interfaces (vtun, switch, pppoe, openvpn)

Please note:
For now, special interface do not support to turn on / off

bar · October 28, 2022, 12:24pm

please check the latest version, I don’t have such interfaces so I have implemented it based on samples provided to me not real data,
If it’s not working well, please open an issue in GitHub with debug log level logs and API / WS samples from the API (documentation available in README)

thanks

dm9991966 · October 28, 2022, 12:59pm

2.0.16 is showing the VLAN switches, but only shows the connected status. The v1.x version also provided the monitored values (send/receive rates, etc.), just like the normal switch0. Would it be possible to make the VLAN switches monitorable as well? Is there anything you need me to provide to help?

And, as others have said, thanks for all your work on this integration!! It’s fantastic!

bar · October 28, 2022, 1:13pm

I need samples as I wrote in the previous post,
In addition, I also mentioned that turn on / off special interfaces (which are not availae out of the box in API) cannot be switched because I need samples of API calls to support it

Thanks

dm9991966 · October 29, 2022, 12:29pm

Bar,
Sorry, didn’t realize the previous post was aimed at me.

FWIW, right now at least, I’m not looking to control the VLAN switches, just get stats on data flow…

I’ve gone through the README, went to the Developer Tools > Services and called the service with the following values:

However, after restarting and looking in the .storage directory I only see the following edgeos.* file

edgeos.5b6595d9b6344fa5ed0d118c6b58ee9d.config.json

Is there something else I need to do to get the debug data to show up? Is there any way I can verify that the service call actually set the settings correctly?

Looking in the above *.config.json, I see the following

    "unit": "Bytes",
    "log-incoming-messages": true,
    "store-debug-data": false,
    "consider-away-interval": 180.0,
    "update-entities-interval": 1.0,
    "update-api-interval": 30

But, no matter how many times I call the service with the above data, the value of store-debug-data never changes.

Sorry for being so dense on this…

WallyR · October 29, 2022, 2:04pm

For these kinds of data you really should look into SMNP instead.
Get the iReasoning MIB Browser and see the data available on the switch, then use NodeRed’s or HA’s SMNP plugins to read out that data.
Just remember that the number for data flow is a rolling 32bit integer for bytes, so whenever it reach its max at 4.294.967.296, then it will start from 0.
This also means you need to extract your values atleast once per ((4.294.967.296*8)/1.000.000.0000=) 34 seconds or you will not know for sure if you had 2 rollovers with full speed or just 1 rollover with a really low speed.

Here are my SMNP flow for NodeRed. It have a few switches of different brands too, but just delete the nodes for the hosts to get rid of them.
My EdgeRouter is an EdgeRouter 4.

[{"id":"d81764c14a65b21a","type":"inject","z":"742cc87db4d12986","name":"","props":[{"p":"timestamp","v":"","vt":"date"}],"repeat":"10","crontab":"","once":true,"onceDelay":"60","topic":"","x":100,"y":585,"wires":[["df6196fbcdc8a9b6","f8e32a5923560d99","861e29b536dc3aad","adad593d0138f83d","2bd20ce8f3574b39","dd31072fb613809b"]]},{"id":"dd9c0114a58cf2c9","type":"ha-api","z":"742cc87db4d12986","name":"sensor","server":"541ade28.b4a62","version":1,"debugenabled":false,"protocol":"http","method":"post","path":"/api/states/{{entity_id}}","data":"","dataType":"json","responseType":"json","outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"results"}],"x":515,"y":765,"wires":[[]]},{"id":"deb440573b10d7a2","type":"snmp","z":"742cc87db4d12986","host":"","version":"2c","timeout":5,"community":"public","auth":"noAuthNoPriv","authprot":"MD5","privprot":"DES","oids":"1.3.6.1.2.1.1.5.0,\n1.3.6.1.2.1.1.6.0,\n1.3.6.1.2.1.1.3.0","name":"basic","x":515,"y":585,"wires":[["bfd8ad8abe116d82"]]},{"id":"97dca3846ee3bd95","type":"function","z":"742cc87db4d12986","name":"make final payload","func":"msg.oidrx=msg.payload;\n\n// The RX and TX values are int32 values and are measuring in bits/s.\n// This means the overflow can happen in just over 17 seconds on some interfaces, if operating at full capacity.\n// The update frequence should therefore not be set higher than 15 seconds to avoid a double overflow in the the timeframe\n\n// Get stored list previous flowstats or initialize it, if it does not exist.\nvar flowstats = flow.get(\"flowstats_\"+msg.oiddevice);\nvar tempvalue;\nvar i;\nif (!flowstats){\n    flowstats = {\n        'timestamp':Math.floor(msg.timestamp/1000)-10 // calculate to seconds\n    }\n    for (i = 1; i <= msg.oidrx.length; i++)\n    {\n        flowstats['if'+i+'_rx']=Math.floor(msg.oidrx[i-1].value/10); // just to give some middle value between 0 and the current start value. Value is bytes/s\n        flowstats['if'+i+'_rx']=Math.floor(msg.oidrx[i-1].value/10); // just to give some middle value between 0 and the current start value  Value is bytes/s\n    }\n}\n\n// Initialize an attribs set for the HA sensor\nvar attribs = {\n    'friendly_name': 'SNMP '+msg.oiddevice,\n    'icon': 'mdi:router-network',\n    'name':msg.oiddevice,\n    'location':msg.oidlocation,\n    'uptime':Math.floor(msg.oiduptime/100) // calculate to seconds\n}\n\nfor (i = 1; i <= msg.oidname.length; i++)\n{\n    attribs['if'+i+'_name']=msg.oidcustomname[i-1].value.toString();\n    if (attribs['if'+i+'_name']==\"\")\n    {\n        attribs['if'+i+'_name']=msg.oidname[i-1].value.toString();\n    }\n}\n\nfor (i = 1; i <= msg.oidenabled.length; i++)\n{\n    attribs['if'+i+'_enabled']=(msg.oidenabled[i-1].value-2)*-1; // 0=disabled, 1=enabled, other=unknown\n}\n\nfor (i = 1; i <= msg.oidlink.length; i++)\n{\n    attribs['if'+i+'_link']=(msg.oidlink[i-1].value-2)*-1; // 0=disabled, 1=enabled, other=unknown\n}\n\nfor (i = 1; i <= msg.oidspeed.length; i++)\n{\n    attribs['if'+i+'_speed']=msg.oidspeed[i-1].value; // Value is bits/s\n}\n\nfor (i = 1; i <= msg.oidrx.length; i++)\n{\n    if (msg.oidspeed[i-1].value==0)\n    {\n        attribs['if'+i+'_rx']=0; // Value in bits/s\n        attribs['if'+i+'_rx_percent']=0;            \n    }\n    else\n    {\n        tempvalue=msg.oidrx[i-1].value;\n        if (tempvalue<flowstats['if'+i+'_rx'])\n        {\n            tempvalue=tempvalue+4294967296;\n        }\n        attribs['if'+i+'_rx']=Math.floor(((tempvalue-flowstats['if'+i+'_rx'])*8)/(Math.floor(msg.timestamp/1000)-flowstats['timestamp'])); // Value in bits/s\n        attribs['if'+i+'_rx_percent']=Math.floor((attribs['if'+i+'_rx']/attribs['if'+i+'_speed'])*100);\n    }\n    flowstats['if'+i+'_rx']=msg.oidrx[i-1].value // Value in bits/s\n}\n\nfor (i = 1; i <= msg.oidtx.length; i++)\n{\n    if (msg.oidspeed[i-1].value==0)\n    {\n        attribs['if'+i+'_tx']=0; // Value in bits/s\n        attribs['if'+i+'_tx_percent']=0;            \n    }\n    else\n    {\n        tempvalue=msg.oidtx[i-1].value;\n        if (tempvalue<flowstats['if'+i+'_tx'])\n        {\n            tempvalue=tempvalue+4294967296;\n        }\n        attribs['if'+i+'_tx']=Math.floor(((tempvalue-flowstats['if'+i+'_tx'])*8)/(Math.floor(msg.timestamp/1000)-flowstats['timestamp'])); // Value in bits/s\n        attribs['if'+i+'_tx_percent']=Math.floor((attribs['if'+i+'_tx']/attribs['if'+i+'_speed'])*100);\n    }\n    flowstats['if'+i+'_tx']=msg.oidtx[i-1].value // Value in bits/s\n}\n\n\nmsg.attribs = attribs;\nmsg.entity_id = `sensor.snmp_`+msg.oiddevice;\nmsg.payload = {\n    data: {\n        state: Date.now(),\n        attributes: attribs\n    }\n};\n\nflowstats['timestamp']=Math.floor(msg.timestamp/1000); // calculate to seconds\nflow.set(\"flowstats_\"+msg.oiddevice, flowstats);\n\ndelete msg.oid;\ndelete msg.attribs;\ndelete msg.oiddevice;\ndelete msg.oidlocation;\ndelete msg.oiduptime;\ndelete msg.oidcustomname;\ndelete msg.oidname;\ndelete msg.oidenabled;\ndelete msg.oidlink;\ndelete msg.oidspeed;\ndelete msg.oidrx;\ndelete msg.oidtx;\n\nreturn msg;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":990,"y":720,"wires":[["dd9c0114a58cf2c9"]]},{"id":"1afa5a5f1b2ced5c","type":"snmp subtree","z":"742cc87db4d12986","host":"","version":"2c","timeout":5,"community":"public","auth":"noAuthNoPriv","authprot":"MD5","privprot":"DES","oids":"1.3.6.1.2.1.2.2.1.2","name":"name","x":515,"y":630,"wires":[["c04edffcedde3131"]]},{"id":"c04edffcedde3131","type":"function","z":"742cc87db4d12986","name":"rearrange","func":"msg.oidname=msg.payload;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":645,"y":630,"wires":[["f904521a37541262"]]},{"id":"f904521a37541262","type":"snmp subtree","z":"742cc87db4d12986","host":"","version":"2c","timeout":5,"community":"public","auth":"noAuthNoPriv","authprot":"MD5","privprot":"DES","oids":"1.3.6.1.2.1.2.2.1.7","name":"enabled","x":780,"y":630,"wires":[["61f07bfeac929664"]]},{"id":"61f07bfeac929664","type":"function","z":"742cc87db4d12986","name":"rearrange","func":"msg.oidenabled=msg.payload;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":960,"y":630,"wires":[["d7cc69cd5610a75d"]]},{"id":"3dc30cea953826ff","type":"function","z":"742cc87db4d12986","name":"rearrange","func":"msg.oidlink=msg.payload;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":645,"y":675,"wires":[["716dccc551da5e1c"]]},{"id":"d7cc69cd5610a75d","type":"snmp subtree","z":"742cc87db4d12986","host":"","version":"2c","timeout":5,"community":"public","auth":"noAuthNoPriv","authprot":"MD5","privprot":"DES","oids":"1.3.6.1.2.1.2.2.1.8","name":"link","x":515,"y":675,"wires":[["3dc30cea953826ff"]]},{"id":"ae349a3b9038a208","type":"function","z":"742cc87db4d12986","name":"rearrange","func":"msg.oidspeed=msg.payload;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":960,"y":675,"wires":[["efeb4a90c9a3ab32"]]},{"id":"716dccc551da5e1c","type":"snmp subtree","z":"742cc87db4d12986","host":"","version":"2c","timeout":5,"community":"public","auth":"noAuthNoPriv","authprot":"MD5","privprot":"DES","oids":"1.3.6.1.2.1.2.2.1.5","name":"speed","x":770,"y":675,"wires":[["ae349a3b9038a208"]]},{"id":"1d59a996087987c2","type":"function","z":"742cc87db4d12986","name":"rearrange","func":"msg.oidtx=msg.payload;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":645,"y":720,"wires":[["761a1aaa3d81c676"]]},{"id":"efeb4a90c9a3ab32","type":"snmp subtree","z":"742cc87db4d12986","host":"","version":"2c","timeout":5,"community":"public","auth":"noAuthNoPriv","authprot":"MD5","privprot":"DES","oids":"1.3.6.1.2.1.2.2.1.10","name":"tx","x":515,"y":720,"wires":[["1d59a996087987c2"]]},{"id":"761a1aaa3d81c676","type":"snmp subtree","z":"742cc87db4d12986","host":"","version":"2c","timeout":5,"community":"public","auth":"noAuthNoPriv","authprot":"MD5","privprot":"DES","oids":"1.3.6.1.2.1.2.2.1.16","name":"rx","x":770,"y":720,"wires":[["97dca3846ee3bd95"]]},{"id":"bfd8ad8abe116d82","type":"function","z":"742cc87db4d12986","name":"rearrange","func":"msg.oiddevice=msg.payload[0].value.replace(\".darkworld.mnt\",\"\");\nmsg.oidlocation=msg.payload[1].value;\nmsg.oiduptime=msg.payload[2].value;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":645,"y":585,"wires":[["730ece6b9f6bd8dc"]]},{"id":"dd31072fb613809b","type":"function","z":"742cc87db4d12986","name":"host=ap1","func":"msg.host=\"10.10.14.21:161\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":290,"y":810,"wires":[["deb440573b10d7a2"]]},{"id":"730ece6b9f6bd8dc","type":"snmp subtree","z":"742cc87db4d12986","host":"","version":"2c","timeout":5,"community":"public","auth":"noAuthNoPriv","authprot":"MD5","privprot":"DES","oids":"1.3.6.1.2.1.31.1.1.1.18","name":"custom name","x":800,"y":585,"wires":[["a2064eb09424bde2"]]},{"id":"a2064eb09424bde2","type":"function","z":"742cc87db4d12986","name":"rearrange","func":"msg.oidcustomname=msg.payload;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":960,"y":585,"wires":[["1afa5a5f1b2ced5c"]]},{"id":"df6196fbcdc8a9b6","type":"function","z":"742cc87db4d12986","name":"host=router","func":"msg.host=\"10.10.12.1:161\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":585,"wires":[["deb440573b10d7a2"]]},{"id":"861e29b536dc3aad","type":"function","z":"742cc87db4d12986","name":"host=switch1","func":"msg.host=\"10.10.14.11:161\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":630,"wires":[["deb440573b10d7a2"]]},{"id":"adad593d0138f83d","type":"function","z":"742cc87db4d12986","name":"host=switch2","func":"msg.host=\"10.10.14.12:161\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":675,"wires":[["deb440573b10d7a2"]]},{"id":"2bd20ce8f3574b39","type":"function","z":"742cc87db4d12986","name":"host=switch3","func":"msg.host=\"10.10.14.13:161\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":720,"wires":[["deb440573b10d7a2"]]},{"id":"f8e32a5923560d99","type":"function","z":"742cc87db4d12986","name":"host=switch4","func":"msg.host=\"10.10.14.14:161\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":765,"wires":[["deb440573b10d7a2"]]},{"id":"541ade28.b4a62","type":"server","name":"Home Assistant","version":5,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"","connectionDelay":false,"cacheJson":false,"heartbeat":false,"heartbeatInterval":"","areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m","enableGlobalContextStore":true}]

mrkaasa · October 29, 2022, 6:22pm

Thanks for the great work. I scratched my head a bit on how to enable monitoring of interfaces/devices on this new 2.0 version.

So just a heads up for anyone new like me, that it is done through the integration screen->EdgeOS Integration->Devices->Device you want to change settings for.

No need to manually extract the attributes anymore, so easy now