Local Deployment for SureFlap / SurePetCare Connect using only local MQTT Broker

eviltom · May 26, 2022, 6:25pm

I get

curl https://hub.api.surehub.io
curl: (7) Failed to connect to hub.api.surehub.io port 443: No route to host

with the docker stopped,

and

curl https://hub.api.surehub.io
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

with it running.

Would I expect to see certificate errors? Pethublocal should be pretending to be the real server at this point?
Or is this because I’m running this from a PC and not from the hub?

curl https://hub.api.surehub.io -v
*   Trying 10.10.30.152:443...
* Connected to hub.api.surehub.io (10.10.30.152) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: none
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: self signed certificate
* Closing connection 0
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Edit:
Just noticed that a http curl gets logge in the pethublocal log, but a https curl doesn’t presumably because its failing the certificate check and going no further. Is there a way to log failed attempts? That would be helpful to see if the hub attempts are making it through.

flyize · May 26, 2022, 6:44pm

add --insecure to your curl command.

eviltom · May 26, 2022, 6:51pm

Docker Running:

curl https://hub.api.surehub.io --insecure
404: Not Found

Docker Stopped:

curl https://hub.api.surehub.io --insecure
curl: (7) Failed to connect to hub.api.surehub.io port 443: No route to host

~~Edit:~~
~~It’s not looking for the root of the domain though; it requests~~

~~>https://app.api.surehub.io/api/me/start~~

~~Which returns~~

curl https://app.api.surehub.io/api/me/start --insecure
{"error":{"message":["Token not provided"]}}

~~but it does that even if the docker is stopped.~~

Never mind, the call to app.api happens later on and only contains profile info. I should be looking at something in the hub subdomin like:

curl https://hub.api.surehub.io/api/credentials --insecure -v
*   Trying 10.10.30.152:443...
* Connected to hub.api.surehub.io (10.10.30.152) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: none
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=*.api.surehub.io
*  start date: Jan  3 04:48:04 2022 GMT
*  expire date: Jan  1 04:48:04 2032 GMT
*  issuer: CN=*.api.surehub.io
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /api/credentials HTTP/1.1
> Host: hub.api.surehub.io
> User-Agent: curl/7.74.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 405 Method Not Allowed
< Content-Type: text/plain; charset=utf-8
< Allow: POST
< Content-Length: 23
< Date: Thu, 26 May 2022 19:29:59 GMT
< Server: Python/3.9 aiohttp/3.8.1
< 
* Connection #0 to host hub.api.surehub.io left intact

So in summary it’s returning 405 Method Not Allowed with a self signed certificate.

flyize · May 26, 2022, 7:40pm

@eviltom Try using this command

eviltom · May 26, 2022, 7:51pm

Yes!

< HTTP/1.1 200 OK
< Date: Thu, 26 May 2022 19:49:38 UTC
< Content-Type: text/html; charset=utf-8
< Content-Length: 64
< Connection: keep-alive
< server: nginx

Ok, so pethublocal is working, but still with that self signed certificate.

flyize · May 26, 2022, 8:31pm

IIRC, the hub doesn’t care that the cert is self signed. So it’s fine.

peterl · May 26, 2022, 8:46pm

I had a very odd issue with Python 3.10 where older ciphers are disabled and had to downgrade to 3.9 to get it working.

Try with curl and openssl s_client

serialnumber=H0xx-0xxxxxx
macaddress=0000xxxxxxxxxxxx
fw=2.43
curl -v -k --ciphers AES256-SHA256 -d "serial_number=$serialnumber&mac_address=$macaddress&product_id=1&firmware_version=2.43" -H "curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3" -H "Content-Type: application/x-www-form-urlencoded" -X POST -o $serialnumber-$macaddress-$fw.bin https://x.x.x.x/api/credentials

or

openssl s_client -tls1_2 -cipher AES256-SHA256 -no_renegotiation x.x.x.x:443

The cipher is the only one supported by the hub (ugh!)

eviltom · May 26, 2022, 9:45pm

That works fine.
OK, so docker seems to be working, responding to requests on 443, but requests from the hub are not getting through. That sounds like a network problem tbh so I’ll spend some time on that.

eviltom · May 27, 2022, 11:30am

Still no nearer unfortunately.
I’ve tried doing some network troubleshooting this morning but I’m not getting anywhere.
I’ve put my Windows box with wireshark in between the hub and the rest of the network and captured what happens.

I see the hub connect and get an IP from windows, do it’s NTP stuff, then query hup.api.surehub.io and is given the correct poisoned result , 10.10.30.152.
After that I’m not really sure. This is a bit out of my depth. It looks like the the server and client start talking (server hello, client hello) and there’s mention of a certificate, but then it all comes to a sudden halt.
This coincides with:

 | Talking to: hub.api.surehub.io | Socket Obtained | Socket Opened | ERROR: wolfSSL_connect failed! Error: -188

Socket Opened | ERROR: wolfSSL_connect failed! Error: -373

Socket Opened | Closing Socket

on the serial.
So it looks like the client doesn’t like the server certificate for whatever reason?
It connects fine if I remove the DNS entry.

Thank you very much @peterl and @flyize for your help with this, and sorry to be a pain in the backside.

flyize · May 27, 2022, 12:11pm

That error -188 suggests that it doesn’t like the self signed cert. That’s not good.

peterl · May 27, 2022, 8:09pm

You are correct @flyize it seems SurePet have released new firmware at some point that checks the certificate. Doing a bit of investigation into it now but if anyone is interested feel free to email me directly (my email is in most of the code and easy to find if you look for it) as I have a few thoughts on this.
Otherwise… Anyone on the new firmware might be stuffed.

flyize · May 28, 2022, 2:15pm

Any possibility of loading the older firmware?

eviltom · May 29, 2022, 10:42am

It’s looking hopeful. The bootloader seems to be unchanged and I managed to get my hub (on the new fw) to force update files served by pethublocal, and with Peter’s help I’ve got the long serial out of the logs and opened the AWS certificate.

flyize · May 29, 2022, 12:42pm

Did you have to crack it open to get the password? I really need to do that.

eviltom · May 29, 2022, 3:03pm

The password was in the serial output as per Hub Certificate | Pet Hub Local but the markers that the parsefirmware function look for were slightly different. I manually edited them to match what it was expecting to find and then the parser worked fine.

flyize · June 1, 2022, 2:46pm

I hope to get a chance to test this soon. It seems that the HA Mosquitto addon doesn’t like zero authentication, so I’ve got to roll my own MQTT server first and get everything switched over.

peterl · June 4, 2022, 7:09pm

Just pushed an update, so now during setup the firmware gets downloaded, and if it isn’t running firmware 2.43 which many of the hubs seem to be upgraded to 2.201 and this version of firmware now checks the certificate. But all hope isn’t lost, I now find the XOR key used to XOR the firmware, and then using a decrypted version of 2.43 I XOR that firmware with the key found in your firmware.

So what does this all mean?? It means you need to flash the old firmware over the top of the hub to make PetHubLocal work and that can be easily done by holding the “reset” button when you power on the hub.

It also means that there is no more need to solder on a console and get the console log to get the Certificate Password, as that is the XOR key used to XOR the firmware with another static XOR key as well. So now the download script also calculates the Certificate password for you too.

D0doooh · June 5, 2022, 2:46pm

i don’t understand it completely, i have set up the docker container, and included it in HomeAssistant via MQTT here it shows me the current status (LOCKEDIN), but i can’t change it?

in the Docker log I see that it wants to do this:

DEBUG:pethublocal:ToHub: Parsed Message [{'pethub/hub/H006-0000000/messages/4E00000000000000': '629cbeed 1000 127 XXX 06'}],
DEBUG:pethublocal:ToHub: Parsed Message [{'pethub/hub/H006-0000000/messages/4E00000000000000': '629cc11a 1000 127 XXX 02'}],
INFO:pethublocal:ToHub: Moving from current lock mode KEEPIN to lock mode UNLOCKED,
DEBUG:pethublocal:Generate Message Hub:H006-0000000 Product:6, Operation:UNLOCKED,
INFO:pethublocal:MQTT: HA Message topic "pethub/ha/H006-0000000_4E00000000000000/KeepIn" message "OFF"

However it is not carried out… what am I doing wrong?

peterl · June 5, 2022, 8:53pm

I assume you are seeing messages from the hub booting and connecting to the stack? As you need to do the DNS poison to make sure the Hub is connecting to the stack.
When the hub boots and everything is working you should see messages like:

[ INFO] : HUB: Inbound Message Topic: "pethub/hub/H0xx-0xxxxxx/messages" Message: "626e685b 0000 Hub online at 1 11 0 43"

If you’re not seeing messages from the Hub because you haven’t poisoned DNS then it’s not going to work.

Also if you are running the new firmware 2.201 which SurePet have been rolling out you will need to downgrade the firmware first. The firmware version is in the pethubconfig.json under the Hub section.

    "Devices": {
        "H0xx-0xxxxxx": {
            "Hub": {
..
                "Device": {
                    "Hardware": "3",
                    "Firmware": "2.43"
                },

If that firmware is 2.201 which many hubs seem to be, then the downgrade process will need to be done first.

flyize · June 6, 2022, 1:11pm

Any chance you’re using the HA MQTT addon and can explain how to get it to work with this?