Changing data flow to only use phone usb0 purely for backup
This solution has been up an running for some time now. I’m happy to report it works well. I put mention in the post above that the wifi access point is only necessary if you have these two requirements.
* I want the phone to run the HA companion application give a wall mounted access point to HA.
* I only want the Nabu Casa connection operation when we're away from the house
If you don’t care about running the HA companion app on the phone, you don’t need to build the wifi access point. Or if you want the companion app on the phone and you’re fine with the Nabu Casa connection being up 24/7 then you don’t need the wifi access point. You could just use the system as described above with outbound internet traffic always going out over the cellular. Or you could create a script that monitors the state of your home internet and only route the traffic over the cellular when your home network can’t talk with the internet. It would a modification of what I’m describing below in this post.
With the system built as described above I hit a point where the wifi access point would drop out for no clear reason. So my system cycle between using my home internet and cellular multiple times a day. Some times it was multiple times an hour. This worked but I didn’t like it. I decided to try different usb wifi adapters in the bridge. Panda wireless controllers are supposed to be the best for working in linux. I had this PAU09 sitting around so I gave it a try. It works great and stopped the wifi from dropping out. It’s a little more expensive and is pretty big with the antennas. I’ve order this as an alternative and expect it’ll do just as well.
Because of the dropouts I also decided I didn’t really need to run all of the internet bound HA traffic through the phone all the time. I made these small changes to the above implementation to only route outbound HA traffic through the phone if my home network had issues accessing the internet.
The first change was to set the HA system so that it configured the usb0 interface with the phone but didn’t bring up the interface. This was done by modifying the usb0 interface specific file located in /etc/NetworkManager/system-connections and setting the autoconnect parameter to false. So this file looks like this for me:
[connection]
id=Wired connection 1
uuid=d021bdf5-afab-357e-b36a-7a162e7a07a9
type=ethernet
autoconnect=false
#autoconnect-priority=-999
permissions=
timestamp=1629971061
[ethernet]
#cloned-mac-address=00:1E:06:42:0C:B5
mac-address-blacklist=
[ipv4]
dns-search=
method=auto
[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto
The next thing was to change the scripts that monitored the status of the wifi provided by the access point. In the initial implementation the script that ran on the wifi access point and the script that runs on HA worked independently. In this new implementation the script that runs on the access point provides the script that runs on HA a notice on the state of the wifi. If the wifi is up and running periodic notifications are sent to the HA script. If the wifi is down then notification are not sent. If the HA script doesn’t receive the notification then it enabled the cellular backup interface.
The script /home/brian/network-watcher/wifi-srv-network-watcher.py that runs on the wifi access point now looks like this
#! /bin/python3
# Program looks to see if it can communicate with the internet, if it can't it takes
# wifi access point down, if it can access interent and wifi was previously taken
# down it brings it back up
import os
import subprocess
import time
import syslog
import socket
import sys
from datetime import datetime
HOST, PORT = "IP_OF_YOUR_HA_CONTROLLER", 5555
connected = False
internet_test_addr = "8.8.8.8" #use google dns YOU COULD CHANGE
full_ping_delay = 5
ping_delay = full_ping_delay
# set so we start WiFi if the internet is up
house_internet_up = False
going_down = False
#Delay a couple seconds for things to come up
time.sleep(2)
syslog.syslog(syslog.LOG_INFO, 'Network Watcher up using target ' + internet_test_addr )
while True:
# check to see if we can ping out internet test address
response = os.system("ping -c 1 " + internet_test_addr + "> /dev/null")
# good ping so internet is up
#print "Ping Response: %d" % ( response )
if response == 0:
#print( 'Good response to ping' )
# if internet was down then bring wifi back up
if not house_internet_up:
#print( 'Good ping ' + internet_test_addr + ' Bringing Wifi Up')
syslog.syslog(syslog.LOG_INFO, 'Good ping ' + internet_test_addr + ' Bringing Wifi Up')
#os.system("rfkill unblock wlan")
response = os.system("systemctl start hostapd")
if response == 0:
house_internet_up = True
syslog.syslog(syslog.LOG_INFO,"wifi successfully activated");
else:
syslog.syslog(syslog.LOG_INFO,"wifi did not activated");
going_down = False
ping_delay = full_ping_delay
try:
#print( 'Trying connection' )
if not connected:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#print( 'Attempt new connection' )
# Connect to server and send data
sock.connect((HOST, PORT))
connected = True
#print( "Connection make")
now = datetime.now() # current date and time
data = now.strftime("%m/%d/%Y %H:%M:%S")
#sock.sendall(bytes(data + "\n", "utf-8"))
sock.sendall(bytes(data, "utf-8"))
#print("Sent: {}".format(data))
except socket.error as msg:
syslog.syslog(syslog.LOG_INFO, "Exception communicating with HA")
connected = False
#sock.close();
# two ping fails so if internet up then take it down
elif going_down:
if house_internet_up:
#print('Cant ping ' + internet_test_addr + ' Shutting down HA WiFi')
syslog.syslog(syslog.LOG_INFO, 'Cant ping ' + internet_test_addr + ' Shutting down HA WiFi')
#os.system("rfkill block wlan")
response = os.system("systemctl stop hostapd")
if response == 0:
syslog.syslog(syslog.LOG_INFO,"wifi turned off");
house_internet_up = False
else:
syslog.syslog(syslog.LOG_INFO,"wifi did not turn off");
ping_delay = full_ping_delay
# first ping fail of internet test address so set to try second ping
else:
syslog.syslog(syslog.LOG_INFO, "First ping failed, look for second error before taking internet down ")
#print "First ping failed, look for second error before taking internet down "
going_down = True
ping_delay = 5
time.sleep(ping_delay)
I changed the name of the companion script that runs on the HA controller from wifi-network-watcher.py to wifi-ctrl-srv.py. I placed the new script in the directory
/usr/share/hassio/homeassistant/os-support-apps/
so that the file file is included in HA backups. Since I changed the script name I had to modify the line in /etc/rc.local that starts the script to match this new name. The script contains the following:
root@HA:/etc/NetworkManager/system-connections# cat /usr/share/hassio/homeassistant/os-support-apps/wifi-ctrl-srv.py
#!/bin/python3
import os
import time
import socket
import syslog
import threading
import subprocess
#from datetime import *
from datetime import datetime
reported_date = datetime.now()
class ThreadedServer(object):
def __init__(self, host, port):
self.host = host
self.port = port
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.sock.bind((self.host, self.port))
def listen(self):
self.sock.listen(5)
while True:
client, address = self.sock.accept()
client.settimeout(60)
threading.Thread(target = self.listenToClient,args = (client,address)).start()
def listenToClient(self, client, address):
global reported_date
size = 1024
syslog.syslog(syslog.LOG_INFO,"Connection from wifi controller")
while True:
try:
data = client.recv(size)
if data:
# Set the response to echo back the recieved data :
response = data.decode("utf-8")
#print("Convert date to time from ",response)
reported_date = datetime.strptime(response, '%m/%d/%Y %H:%M:%S')
#client.send(data)
else:
raise error('Client disconnected')
except Exception as e:
syslog.syslog(syslog.LOG_INFO,"Exception in client that talks with wifi contoller " + str(e))
client.close()
return False
def ha_up():
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("localhost", 8123))
s.close()
return True
except socket.error:
return False
def send_mqtt_msg( msg ):
# only continue if mqtt broker is up
done = 1
while done != 0:
#print("Sending message",msg)
done = os.system( msg )
if done != 0:
time.sleep(5)
def control_thread():
global reported_date
usb0_down = True
wifi_down = False
loop_delay = 5
while not ha_up():
syslog.syslog(syslog.LOG_INFO,"Waiting on HA to come up")
time.sleep(10)
syslog.syslog(syslog.LOG_INFO,"Ha up!, let's watch wifi state")
# send initial message indicating wifi is on
send_mqtt_msg("mosquitto_pub -u mqtt -P Hass10mqtt -h 127.0.0.1 -t home-assistant/wifi -m "+ '{\\"state\\":\\"ON\\"}')
refreshTime=0
while True:
cur_date = datetime.now()
time_diff = cur_date - reported_date
#print( "Time difference is ",time_diff.total_seconds())
if time_diff.total_seconds() >= 30:
wifi_down = True
if usb0_down:
syslog.syslog(syslog.LOG_INFO,"need to bring up usb0 cellular backup interface")
# Let HA know system switching from wifi to cellular
send_mqtt_msg("mosquitto_pub -u mqtt -P Hass10mqtt -h 127.0.0.1 -t home-assistant/wifi -m " + '{\\"state\\":\\"OFF\\"}')
#Add command to bring usb interface up
usb0_status = subprocess.check_output("nmcli con up 'Wired connection 1'", shell=True, text=True)
if "successfully activated" in usb0_status:
usb0_down = False
syslog.syslog(syslog.LOG_INFO,"usb0 successfully activated");
else:
syslog.syslog(syslog.LOG_INFO,"usb0 did not activated");
else:
wifi_down = False
if not usb0_down:
syslog.syslog(syslog.LOG_INFO,"need to take down usb0 cellular backup interface")
send_mqtt_msg("mosquitto_pub -u mqtt -P Hass10mqtt -h 127.0.0.1 -t home-assistant/wifi -m " + '{\\"state\\":\\"ON\\"}')
#Add command to take usb interface up
usb0_status = subprocess.check_output("nmcli con down 'Wired connection 1'", shell=True, text=True)
#print("usb0 status",usb0_status);
if "successfully deactivated" in usb0_status:
usb0_down = True
syslog.syslog(syslog.LOG_INFO,"usb0 successfully deactivated");
else:
syslog.syslog(syslog.LOG_INFO,"usb0 did not deactivated");
refreshTime = refreshTime + loop_delay
if refreshTime >= 60:
refreshTime = 0
if wifi_down:
send_mqtt_msg("mosquitto_pub -u mqtt -P Hass10mqtt -h 127.0.0.1 -t home-assistant/wifi -m " + '{\\"state\\":\\"OFF\\"}')
else:
send_mqtt_msg("mosquitto_pub -u mqtt -P Hass10mqtt -h 127.0.0.1 -t home-assistant/wifi -m " + '{\\"state\\":\\"ON\\"}')
time.sleep( loop_delay )
if __name__ == "__main__":
#print("In main code kicking off control_thread")
ctl_thread = threading.Thread(target=control_thread)
ctl_thread.start()
#print("In main code kicking off socket server")
# start service to listen for time information
ThreadedServer('',5555).listen()
I think this is a much cleaner implementation.
Migration to Debian Bullseye Notes:
In the script above there are two “nmcli con” lines, one for up and one for down. The name ‘Wired connection 1’ is the name of the NetworManager file in “/etc/NetworkManager/system-connections”. I recently moved over to Debian Bullseye and the file name in that directory for the USB device became “Supervisor usb0.nmconnection”. As a result I had to change the line lines in the script from ‘Wired connection 1’ to ‘Supervisor usb0’.
When moving to Debian Bullseye, even though I have eth0 defined in /etc/network/interfaces, NetworkManger was doing some management for the interface. I didn’t want this. To make sure NetworkManager would not manage the interface I created the file ‘/etc/NetworkManager/conf.d/99-unmanaged-devices.conf’ containing the line
[keyfile]
unmanaged-devices=interface-name:eth0