Yeah, I have all the stuff, I was just hoping that I might be able to temporarily tape the connections down. That looks pretty tiny tho.
I know someone who did manage to tape it down and do the flash as the messages on the console come up in the first 20 seconds so it doesn’t take long. I personally thought they were quite keen doing it. But managed to do it, taking it off the mount as shown in photo would be easier but depending on where you live you might be able to find someone who can help.
1 Like
I managed it using these to clip onto the pads. It worked a treat, unfortunately I only had two so used a crocodile clip to make the third connection… it slipped and toasted the hub power circuitry! I investigated and found that I could plug a micro usb into the bottom of the hub, that bypassed the normal supply input to the hub and it came back to life working 100%. The PCB clips work well as long as you take care!
1 Like
Just pushed some changes to move to having unit tests for pethublocal so regression testing any of the changes should be far simpler for the cat flap now. Please report any crashes weirdness I missed either here or email me directly with traces.
1 Like
Long time no public updates… I have been emailed by a few folks and I thought I would do a status update as to what I have been up to. The project isn’t dead it’s just work and family commitments have come first and this fits around everything else.
But what I have decided that it’s overly complicated to deploy and support so I am refactoring most of the code to vastly simplify deploying it and monitoring it.
Some of the changes so far include:
- Having a “server” browser UI to let you do infrequent steps like adopting new devices
- Changing the curfews will also be in the server UI as I was trying to support it in HA but it all just got too difficult for me to setup and maintain if I wanted to only use MQTT discovery and it was rapidly leading me down the path of creating a full HA component for it, which I didn’t really want to do as ideally I am wanting the stack to be able to work with other open source home automation products like Openhab.
- Collapsing all code into flask apps one for http and another for https and using the flask mqtt client rather than paho. This means that everything is now in flask and I can use socketio to send messages from the server to the browser. Lots of opportunities open up from doing that. Also allows having dynamic creation of time controls for curfews on the browser side.
- Unit tests for all the messages, as I now have every surepet device type including the cat flap and the felaqua (Thanks Toby!) it means I have all the messages in both directions covered. Still have a few messages to figure out especially the RSSI value which comes from the hub. But 99% of the other messages are now worked out and documented in the code.
- Working on catching all random errors to make sure it doesn’t crash when dictionary values are blank. I know this has been an issue in the past and shouldn’t be going forward
- Database loaded as a in-memory dictionary. Means all processing is now much faster and only update the database every minute when things change rather than needing to do a request each time I need to map a tag to a animal.
- Moving to using the main host MQTT broker, as this was annoying for some folks having a broker in docker and then sending all messages to the main broker. Now I set credentials with the broker username/password so the hub itself can login to your main broker if you have username/password on there and the flask mqtt client uses the same password to authenticate.
- End goal is one docker container for “the app” or being able to run it outside docker with two processes one for http and the other for https for the credentials API call.
- Lots of other improvements in documentation and readability.
When it’s ready to go I will commit the code, but it is still very much work in progress right now. Anyone interested in helping cut the code if you have time feel free to email me offline.
2 Likes
Thank you for your hard work! I finally bought an hub v8 model. At the moment my girlfriend also wanna use the official cloud app. Is it a good idea to read out the password before connecting it to the internet? I wanna store the password for the future if i wanna use your local solution.
poah 
tl;dr? 
1 Like
If it is a brand new “v8” version never connected to the internet before. Yes all you need to do is power it on isolated from the internet pointing to the docker stack, and you should see a “long_serial” being sent to the hub.api.surehub.io the first time it connects, that is the password required to decrypt the certificate.
Otherwise you can do the firmware update replacement as long as you connect a TTL Serial cable on, as that works irrespective of the current version loaded. But does require opening the hub and soldering onto it. Which if you mess it up, could blow up the hub.
I do know some python from my web scraping days and would love to help, but would probably make even more work for you by asking a ton of basic questions. 
Sorry for my noob skills, if i capture the password with my never connected v8 hub, write it down for future use.
If i then connect the hub to the internet and use the cloud app, will i be able to revert back at some point using the old pasword and use your local solution? (or is the password changed due firmware updates?)
I think you’re misunderstanding what the password is used for @joostnl.
When the hub boots it makes a POST over HTTPS to the /api/credentials endpoint to download what I call the credentials. In the response back from the surepet cloud service is a Base64 encoded PKCS12 Certificate that the hub then decrypts using the password you now have.
That certificate it used when the hub connects to the SurePet AWS IoT MQTT endpoint using Mutual TLS to authenticate the hub to AWS. This is a two way handshake so the traffic is encrypted and there is normally no way to inspect the traffic… But that now isn’t the case as you have the certificate password.
What it means is you can use PolarProxy as a man in the middle proxy in between the Hub and the cloud service. If you spoof the Surepet AWS DNS host a5kzy4c0c0226-ats.iot.us-east-1.amazonaws.com locally as well to point to PolarProxy, then PolarProxy uses the certificate extracted from the /api/credentials file and the password you now have to open the PKCS12 certificate file for PolarProxy to initiate the connection to the AWS IoT MQTT endpoint.
So the traffic gets decrypted by PolarProxy, saved to a pcap file that you can load into Wireshark whenever you want and see the decrypted traffic in both direction, then re-encrypts the traffic and connects to AWS. So the hub and AWS knows no difference as they are sending all the same messages to each other, you just happen to be decrypting it in the middle and can now see all the messages that the Cloud service sends back to the Hub.
That is what this line in the shell script is doing:
./PolarProxy -v -p 8883,1883 --autoflush 10 -o mqtt --insecure --clientcert a5kzy4c0c0226-ats.iot.us-east-1.amazonaws.com:$serialnumber.p12:$certpassword --servercert a5kzy4c0c0226-ats.iot.us-east-1.amazonaws.com:iot.p12:password --nosni a5kzy4c0c0226-ats.iot.us-east-1.amazonaws.com
What the above means is the PolarProxy listens on 8883 which is the encrypted MQTT port, and saves traffic as if it was port 1883 which is unencrypted MQTT traffic. The --clientcert where it assigns the hostname you are connecting to, with the serialnumber.p12 which is the PKCS12 file. Then uses a self signed certificate as the server certificate as the hub doesn’t check on HTTPS or MQTT if the certificate is legitimate (please don’t change this Sure Pet Care… it would break all my work!) and then connects off to AWS.
Does that make sense?
Just looking through the javascript and it seems the Felaqua had a internal product name of Poseidon, you folks missed a trick as Poseidon is a way cooler name than Felaqua.
https://surepetcare.io/assets/images/product/poseidon-render.png
2 Likes
Thank you Peter for your detailed post, I only think it is too much for me as simple use to do.
Even if i get it to work the benefits are not that much.
I think i will run the offial cloud app and if that is end of life i will use your local solution with a little soldering.
Hi @peteri,
I wonder if you can point me in the right direction. I just want to make sure i have got my head around things properly and have a few questions, if you do not mind.
- I use Docker via my Synology NAS. Can you see any reason this could be an issue?
- Are there any hardware requirements if you are just want local control via MQTT.
- Where does the local DNS update for need to forward to?
I did try and install the containers using docker compose but hit an issue as below
Starting mqtt ...
Starting web ...
Starting web ... error
WARNING: Host is already in use by another container
Starting mqtt ... done
ERROR: for web Cannot start service web: driver failed programming external connectivity on endpoint web (c14a578b440feca4a7bb9fc07cabe6fc1525baf7fd86ac2fcae2fd3d0070bc62): Error starting Starting pethub ... done
Starting msgs ... done
ERROR: for web Cannot start service web: driver failed programming external connectivity on endpoint web (c14a578b440feca4a7bb9fc07cabe6fc1525baf7fd86ac2fcae2fd3d0070bc62): Error starting userland proxy: listen tcp4 0.0.0.0:443: bind: address already in use
ERROR: Encountered errors while bringing up the project.
admin@hub:/volume1/docker/pethublocal$
Any help or advice would be really appreciated.
Thanks
The issue you will face on a Synology is that you need to listen on port 443 for the HTTPS webserver component, and the host itself has an nginx instance running on port 443 to redirect to port 5000 / 5001.
The other port you need is TLS over MQTT on port 8883, which you will most probably have open.
grep -B1 443 /etc/nginx/nginx.conf
server {
listen 443 default_server ssl;
listen [::]:443 default_server ssl;
Because nginx and the NAS itself is using that port you can’t deploy docker to use that same port.
It’s easier to run on a Pi or any other small computer able to run linux that you can free up port 443 and 8883, or the other option is you could only poison DNS for only the AWS endpoint but then you will need to have the docker MQTT stack listening on a different topic.
I’ve been writing a bit of documentation about a month ago but had to park it due to work commitments, but hope to pick it up this week and into the new year over the Christmas break.
Hey @peterl,
I think I am very close but just cannot work out what I have may have missed or not done in the process.
I have setup my MQTT server for Home Assistant with the cert files etc. This seems to all be OK and I am getting the correct response on an nslookup for the API endpoint.
I have also setup my Raspberry Pi to run Pet Hub Local. The only issue I seem to get is shown below.
Starting web ... done
Starting mqtt ... done
Starting pethub ... done
Starting msgs ... done
Attaching to pethub, web, mqtt, msgs
mqtt | 1640960057: Warning: Mosquitto should not be run as root/administrator.
mqtt | 1640960057: Connecting bridge pethublocal (192.168.1.20:1883)
mqtt | 1640960060: New connection from 172.18.0.4 on port 1883.
mqtt | 1640960060: New client connected from 172.18.0.4 as auto-7D6BB789-3DB7-34DF-496D-2F20328B04A5 (p2, c1, k60).
mqtt | 1640960060: auto-7D6BB789-3DB7-34DF-496D-2F20328B04A5 0 pethublocal/#
pethub | Init :HAMQTTIP environment: 192.168.1.20
pethub | Init :HAMQTT Host: 192.168.1.20
pethub | Init :HAMQTT Port: 1883
pethub | Starting Pet Hub Local
pethub | Init :Connecting to Home Assistant MQTT endpoint at 192.168.1.20 port 1883
pethub | Database:SQLite init database response: {'pets': [{'name': 'Tigger', 'species': 1, 'device': None, 'product_id': None, 'state': None}]}
pethub | Database:Load Devices from pethublocal.db and create Home Assistant MQTT discovery config topics
pethub | Traceback (most recent call last):
pethub | File "/usr/local/lib/python3.8/site-packages/box/box.py", line 488, in __getitem__
pethub | return super().__getitem__(item)
pethub | KeyError: 'devices'
pethub |
pethub | The above exception was the direct cause of the following exception:
pethub |
pethub | Traceback (most recent call last):
pethub | File "/usr/local/lib/python3.8/site-packages/box/box.py", line 514, in __getattr__
pethub | value = self.__getitem__(item, _ignore_default=True)
pethub | File "/usr/local/lib/python3.8/site-packages/box/box.py", line 509, in __getitem__
pethub | raise BoxKeyError(str(err)) from _exception_cause(err)
pethub | box.exceptions.BoxKeyError: "'devices'"
pethub |
pethub | During handling of the above exception, another exception occurred:
pethub |
pethub | Traceback (most recent call last):
pethub | File "/usr/local/lib/python3.8/site-packages/box/box.py", line 516, in __getattr__
pethub | value = object.__getattribute__(self, item)
pethub | AttributeError: 'Box' object has no attribute 'devices'
pethub |
pethub | The above exception was the direct cause of the following exception:
pethub |
pethub | Traceback (most recent call last):
pethub | File "pethubmqtt.py", line 403, in <module>
pethub | for device in pethubinit.devices:
pethub | File "/usr/local/lib/python3.8/site-packages/box/box.py", line 530, in __getattr__
pethub | raise BoxKeyError(str(err)) from _exception_cause(err)
pethub | box.exceptions.BoxKeyError: "'Box' object has no attribute 'devices'"
web | SUREHUBIO from environment set in config.ini
web | DOWNLOADFIRMWARE from environment set in config.ini
web | * Serving Flask app 'app' (lazy loading)
web | * Environment: production
web | * Serving Flask app 'app' (lazy loading)
web | WARNING: This is a development server. Do not use it in a production deployment.
web | * Environment: production
web | Use a production WSGI server instead.
web | * Debug mode: off
web | WARNING: This is a development server. Do not use it in a production deployment.
web | Use a production WSGI server instead.
web | * Debug mode: off
web | * Running on all addresses.
web | WARNING: This is a development server. Do not use it in a production deployment.
web | * Running on all addresses.
web | WARNING: This is a development server. Do not use it in a production deployment.
web | * Running on http://172.19.0.2:80/ (Press CTRL+C to quit)
web | * Running on https://172.19.0.2:443/ (Press CTRL+C to quit)
pethub exited with code 1
At this second, I have not got it working but i feel i am very close. Is there anything above which might get me on the right path? Seems an issue around “devices” on the pethublocal but i just can not figure it out.
I wonder did you create the database using the make pethublocal database script?
As you should have a pethublocal.db sqlite database, and in there should be the download of the start json file converted into the database tables based on your configuration. Feel free to send me an email offline and I can check out your start json file to see what is wrong.
Any updates on this?
As I understand, this project is designed to be a dns-spoofed local replacement of the surepetcare.io services right?
Is this an easier approach than trying to replace the hub itself (either with a Pi+MiWi module or custom firmware) to make the Connect Hub offer a local API?
Im also a little disappointed how often SurePetCare API seems to be down rendering my pet flap useless (as I dont use curfews, but control opening/closing via HA) and thus im looking for a way to make it work locally.
This reminds me. The documentation isn’t clear to me for grabbing my password. What are the side connector pins? The docs say this:
I recommend soldering to the side connector to pins 3, 7 & 8 as per:
pethublocal/docs/Hub at main · plambrechtsen/pethublocal · GitHub
However, those same docs have two charts with some of the same pin numbers. Also, I see no mention of power, only ground. Does this mean its actually plugged in during this process?