I am looking for a way to monitor network events such as specific types of RESTful calls made when publishing weather data to many diufferent destinations (let’s use this example). What I would like to do is to be able to see in my dashboard a list of destinations and next to destination, the most recent time a certain kind of messagte was sent to that destination (the destinations are all known). I could try to capture this at the source but my problem is some of these messages come from many different sources (going to the same destination) within my network. This way I can be alerted if data is not sent when it should have been. Any kind of a flexible network traffic measuring / monitoring tool whose results can be exposed within HA to be able to show related statistics or trigger automations would be outstanding. Weather data is just one example. I have several scenarios where I need to be able to see or be alerted about different kinds of network traffic depending upon the type/content of the packet as well as source and destination. Any ideas?
@KruseLuds I like your idea, and I have a similar request that might be inline with your ask, but with more functions involved? Let me know if this sounds comparable to what you are wanting to do. Currently I use a tool called sysmon:
https://puck.nether.net/sysmon/
on a Linux box to trigger bash scripts that cycle X10 devices (reboot routers, WAPs, Pis, etc.). I am considering trying to figure out how to compile it on my raspberry pi 4 inside the HA install. It is a bit beyond my current skillset, because I have never looked at Docker, but I am pretty comfortable with Linux, and have other Pis that I have gone thru insane lengths to get software running on them. I do not want to re-invent the wheel, so if there is a similar tool already I would use it.
With all that said, being able to just copy my sysmon.conf file to the HA instance would be a plus. I am swapping out my X10 devices with smart plugs, so I want to ping or do http “gets” on my network gear, then trigger on/off events to the smart plugs based on any timeouts. Displaying the sysmon.html page could probably be easily placed in a div or an iframe?
I like your idea about sysmon. As a related item, let me say I did resolve part of my issue but it was a real hack. Some of the network traffic in which I am interested is on the same Raspberry PI, which publishes some of the weather data to the web for me. I found out about HA after I had a pretty complicated process with a web server and a bunch of other weather related data that is gathered and analyzed and stored in a DB before transmitting results every 5 minutes to send sites all over the place. That is all done via software called weewx running on the raspberry pi.
Because of that I was unable to install the Home Assistant version that runs it’s own OS, as that would blow away weewx, so I installed HA after much struggle and retries onto the RPI in docker.
So, my hack for some of the network traffic to show in HA, was I created a bunch of command_line sensors with very complex value_templates to get the formatting just right, then displayed those sensor values in HA. The struggle was to figure out how to use grep in those sensors to get the data from tailing certain data in the weewx logs and the syslog in the RPI. As anything in a docker cannot by design see anything outside of that docker, I discovered a hack on YouTube showing me how to share a directory on the rpi with a docker instance, and I shared my log directories to it. It does actually now work very well but it was a headache to figure out. Also, I am stuck with a situation now where almost every time the RPI or Home Assistant have an update to install, I do go ahead and install it but 90% of the time then that shared directory is no longer available in docker, and I have to go through the same 15 or so steps to get that directory shared into the docker again.
A shared directory can be specified within a docker instance, and the proper way to do it is to include that setup when the docker is first created. However, that would force me to completely have to reinstall the HA setup I have on docker. I do plan on doing that soon, but it would not be as much work as you think, because I have an add-on that backs up my entire HA setup locally and to the cloud every couple of hours. I fortunately I have had to restore one of those when I did something to break everything - I would restore the entire RPI from a recent backup, then go into HA and restore the HA setup that had been backed up so not much is lost.
Since I boot off of a 1TB SSD on the RPI rather than an SD Card (the performance of the RPI in almost every respect is then VASTLY better), it is really a nightmare to create a shrunken backup that can be readily installed onto an SD card, but I have figured out how to do that. When an RPI boots off of any drive, the debian OS automatically uses the entire drive (or sdcard), so I had to buy a 2tb drive for my PC to be able to even make a copy of the SSD, to manipulate and shrink to an image that can fit onto an SD card. I have the steps documented if you need them, as well as the steps for installing ha in a docker (but not with the shared directory!).
I have heard of sysmon and have used it for windows, it is terrific.
Let’s keep digging for a network monitoring tool that is a local integration or add-on for HA (meaning it doesn’t come from a vendor that you pay for their service and the vendor needs to see your traffic or data - I have seen those integrations and say “no thank you”!).
Let me know what you think, I am sure we can come up with more elegant solutions…
it sounds like you and I have had a similar journey?!
I have 9 raspberry pis and a couple of NUCs running various services, and a very complex homegrown Linux-based x10 home automation system with a custom web interface that I wrote over a period of 15 years…
I have my own weather station that publishes to weather underground, I used to have a Cronned bash script that I did a wget of the WU page and use grep and awk to dump the temp and humidity to a log file. I ran MRTG to graph the data, and I would launch scripts within MRTG to Control my x10 thermostat and sprinklers.
now I also use weewx, and I am moving away from X10, I am incorporating Home Assistant on a newly provision raspberry pi4. I have swapped out all my X10 switches and receptacles with lutron Caseta devices. and that was pretty painless, so I’m moving ahead with an ecobee, and possibly a rachio sprinkler controller (I want a rainmachine, but they are out of stock forever?!)
and I think the sysmon you are referencing is from sysinternals, and is nothing like the compiled c sysmond I am using…
The sysmon I am using is a pretty basic Network monitor daemon that looks at various TCP end-points and creates an index.html file With the table of the data. It can also be ran from the command line and output to standard out. One of the cool features is that when one of the endpoints times out it can launch a script. so right now when I can’t ping my ISP gateway, I can launch a script that reboots the X10 receptacle that my ISPs router is plugged into every time it fails to ping the gateway three times. I have it set to poll every five minutes. there are numerous other uses, I also have a script that upon not being able to reach a webpage after five minutes, it can reboot Apache. it will also send me a text to let me know that somethings wrong. it is a very lightweight daemon, and I’ve been using it since 2014. It hasn’t really been updated since then, and I have modified the source to use CSS in the html output, since the original source was pretty basic.
If you have a Linux VM available, you should compile it and check it out. If you are at all a network geek, like I am, I think you will find it pretty useful. I haven’t had the time since I learned a bit of Python, but porting it to Python has been on my to do list for about a year now. 
if I figure out a solution to my needs I’ll definitely post it here. cheers!
Thank you please do!
Any update?
I am still trying to get all my x10 stuff switched out, so haven’t had a lot of time. Briefly looked at Uptime Kuma, but have not really dug deep. Currently, I am trying to find an easy solution to securing the login, and get the newly installed Rachio system working. I need to get a Thermostat and figure out a solution for the Garage door, and then I will take a deeper dive into monitoring. Right now I am using sysmond still on a separate box, until I learn more about HA and it’s inner workings.
I have limited time with everything else going on, so it might be a month or so before I can try to implement something, but it is on the to-do list.
OK… Had a few minutes to look at some videos. This will Interest you:
If you have already looked at Uptime Kuma or Installed it, check out the HACS Integration to bring it into HA and create Automations starting at 5m 58s in the video…
I think this is the solution I will be using. 
The part to pay close attention to is the HACS integration getting installed on top of the core add-on. The HACS integration brings in some addition sensors that allow scripting and displaying info on HA dashboard. I have a monitor on my gateway and if it fails 3 times, a HA script resets a smart plug that my router is plugged into. 
Nice! Thank you for letting me know about this! (I do alrteady have HACS installed in my supervisor instance). But how does this monitor specific network traffic?
I guess it depends on how granular you want to get… If you’re looking to scrape websites beyond just a keyword and possibly store values as variables: I don’t know that it’ll allow you to go that far. However, as far as network monitoring goes, it has a pretty complete list of monitor types…
and with the HACS integration, it allows you to trigger automations and scripts on sensor states. I currently have it rebooting my gateway router (via smart plug) if a ping fails three times. I need to figure out how to issue a command through SSH to restart services (Apache, nginx, mail, etc) on Remote hosts on failure. If I can accomplish restarting services, it can definitely replace my aged sysmond set up. 
An entities card allows you to display the monitor states in a dashboard, so it works for my needs; Display monitor state, notifies, and provides automated task to try to recover state on failure.