Unfortunately, no, I don’t have the CC2531 adapter.
I have an unused “Zigate” dongle (based on NXP JN516X) and it has also a sniffer firmware available, but I’m not sure if I can use it to get a capture file compatible with your analyzer…
Anyway, I’ll use the docker stack to make additional captures of the cat flap frames…
K.
It looks like this guy might have made some compatible firmware. All I need is a wireshark style pcap file. Or even just a text based output with the data frames in a hex stream.
The protocol is MiWi aka pseudo Zigbee but really just standard 802.14.5 long frames and then a 08 beacon frame and 01 data frame with an encrypted payload. It’s the data frames I am interested in as then I run the dexor over it and know what is going on.
@peterl i just received my sureflap (type: impdwt without the hub)
Would it help you if i set up my raspi 2b with my cc2531 to sniff for you?
Or will i only be able to sniff with an hub
Hi @joostnl in my view you should just go out and buy a Hub as well. Yes technically you can build the wemos radio hub solution I have posted or go for Mark’s option of using a Pi and beeclick but now with what I have reverse engineered with the hub and running it locally I would just stick with the hub that surepet supply as it just works with the door hardware and you can completely cut over the hub to work locally with my docker stack.
The only downside to the hub is you require an Ethernet port rather than it working over wifi which the wemos will do. So you have some more flexibility there.
But if you feel like improving my wemos code or altering Marks code to the point it’s completely functional with all the radio messages which will require a bit of work then that would be fantastic.
It’s in my backlog of activities for the moment as I now have just sending messages to the hub as steps to complete and the messages required have for the most part been fully decoded… I just need to document that. But 99% of what I have figured out is in the pethubpacket.py file.
Few more updates to the pethubmqtt parsing and moved it to use paho mqtt as it was just easier to develop with and also configured it to use my home assistant mqtt instance, and setup the docker compose to have a broker to have pethublocal/# in both directions as it just makes developing support for both directions that much easier.
ok so something that caught me out while trying the docker stack with the normal hub.
Should that point to the local_mqtt container or to the mqtt server that Home Assistant connect to?
Yeah I need to improve the documentation :)…
That should be pointing to the home assistant MQTT endpoint.
And this line in the mosquito.conf needs to point from the hub mqtt to your home assistant mqtt as a broker connection.
Time for documentation I think. 
Edit… Pushed some updates as I had broken things in pethub for not importing the correct packages and corrected how the docker compose for pethub was building. It works now
Adding documentation on how it all works here:
Awesome thank you.
Yeah I figured out paho-mqtt on my own. Haven’t updated to the last commit yet, but I don’t see the hub connecting to my local stack.
I can see it is lookup the the DNS:
But I don’t see the connection on the http log. I do see when I test with curl. Took me a while to figure out how to make traefik do tcp forwarding (so hope it works). Unless the hub doesn’t support working with SNI then I really have to move it I think.
# curl https://hub.api.surehub.io/ -k -v
* Trying 192.168.241.12...
* TCP_NODELAY set
* Connected to hub.api.surehub.io (192.168.241.12) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=hub.api.surehub.io
* start date: Jan 18 08:52:09 2021 GMT
* expire date: Jan 1 08:52:09 2032 GMT
* issuer: CN=hub.api.surehub.io
* SSL certificate verify result: self signed certificate (18), continuing anyway.
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET / HTTP/1.1
> Host: hub.api.surehub.io
> User-Agent: curl/7.58.0
> Accept: */*
>
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/1.1 200 OK
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< Content-Type: text/html; charset=utf-8
< Content-Length: 37
< Server: Werkzeug/1.0.1 Python/3.7.10
< Date: Mon, 05 Apr 2021 06:18:53 GMT
<
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* Connection #0 to host hub.api.surehub.io left intact
The hub doesn’t send SNI when it attempts to connect, so the docker compose needs to listen on a real IP not run through a virtual service.
I added a second IP address on my Pi running it and made sure that any existing web servers I had listening on 443 were bound to the main IP.
Then in your docker compose comment out 443 and uncomment the binding with the IP.
#Flask python web server that listens on port 443 for the hub to connect to on boot, also downloads the creds on the first attempt from the cloud service.
web:
build: web
container_name: local_web
volumes:
- './output/web:/web/creds'
ports:
- "192.168.20.241:443:443"
# - "443:443"
If it is working you will see something like:
Post payload : {"serial_number": "H0xx-xx", "mac_address": "xx", "product_id": "1", "firmware_version": "2.43"}
192.168.1.167 - - [03/Mar/2021 07:13:59] "POST /api/credentials HTTP/1.1" 200 -
So I think keep it simple and either shutdown all listening services on 443/8883 on the main host, or add a secondary IP to the host and bind to that.
Added support for pet door state as a switch rather than a sensor. Seems like HA MQTT Discovery doesn’t support value_templates too well so I couldn’t have a single state / command value and figure out the switches accordingly. So now have an inbound and outbound switch to display state. Tomorrow I will add the set.
ok not sure what I am missing now.
I don’t see anything pushed into HA, but I do see messages logged.
This is what pethub is doing:
local_pethub | Starting Pet Hub
local_pethub | Connecting to Home Assistant MQTT endpoint at comms.heaven.za.net port 1883
local_pethub | {'devices': [('Home hub', 1, '0000801F121D6719', 4, 0, None, None, None, None, None, None, None, None), ('Kitchen', 3, 'D98122FEFFEC1054', None, None, 0, '12:01', '12:02', 2, None, None, None, None)], 'pets': [('Ziva', 2, 'Kitchen', 3, 1), ('Jessie', 2, 'Kitchen', 3, 1)]}
local_pethub | Load Devices from pethublocal.db and init Home Assistant MQTT discovery configuration
local_pethub | Loading Hub: ('Home hub', 1, '0000801F121D6719', 4, 0, None, None, None, None, None, None, None, None)
local_pethub | Loading Pet Door: ('Kitchen', 3, 'D98122FEFFEC1054', None, None, 0, '12:01', '12:02', 2, None, None, None, None)
local_pethub | data published 1
local_pethub | data published 2
local_pethub | data published 3
local_pethub | data published 4
local_pethub | Door state 2
local_pethub | data published 5
local_pethub | data published 6
local_pethub | data published 7
local_pethub | data published 8
local_pethub | Load Pets from pethublocal.db and init Home Assistant MQTT discovery configuration
local_pethub | Loading Pet: Ziva for feeder Kitchen
local_pethub | data published 9
local_pethub | data published 10
local_pethub | Loading Pet: Jessie for feeder Kitchen
local_pethub | data published 11
local_pethub | data published 12
local_pethub | Subscribe to pethublocal and home assistant topics
local_pethub | OnMessage pethublocal/messages/D98122FEFFEC1054 1 b'606b141a 07c0 132 117 321 6 01 0c 01 0c 02 01'
local_pethub | OnMessage pethublocal/messages/D98122FEFFEC1054 1 b'606b141a 07d0 132 118 36 1 04'
local_pethub | OnMessage pethublocal/messages/D98122FEFFEC1054 1 b'606b141b 07e0 132 119 40 1 02'
local_pethub | OnMessage pethublocal/messages/D98122FEFFEC1054 1 b'606b141b 07f0 132 120 519 6 02 0c 01 0c 02 01'
local_pethub | OnMessage pethublocal/messages/D98122FEFFEC1054 1 b'606b141e 0800 132 121 519 6 01 0c 01 0c 02 01'
local_pethub | OnMessage pethublocal/messages/D98122FEFFEC1054 1 b'606b141e 0810 132 122 36 1 00'
local_pethub | OnMessage pethublocal/messages/D98122FEFFEC1054 1 b'606b141e 0820 132 123 40 1 02'
This is msgs:
local_msgs | Apr 05 15:28:47 pethublocal/messages/D98122FEFFEC1054 606b108c 0650 132 99 483 6 01 0b 2d 0c 03 01
local_msgs | Apr 05 15:28:47 pethublocal/messages/D98122FEFFEC1054 606b108c 0660 132 100 489 6 01 0b 2d 0c 03 01
local_msgs | Apr 05 15:28:47 pethublocal/messages/D98122FEFFEC1054 606b108c 0670 132 101 495 6 01 0b 2d 0c 03 01
local_msgs | Apr 05 15:28:47 pethublocal/messages/D98122FEFFEC1054 606b108c 0680 132 102 501 6 01 0b 2d 0c 03 01
local_msgs | Apr 05 15:28:47 pethublocal/messages/D98122FEFFEC1054 606b108c 0690 132 103 507 6 01 0b 2d 0c 03 01
local_msgs | Apr 05 15:28:47 pethublocal/messages/D98122FEFFEC1054 606b108c 06a0 132 104 513 6 01 0b 2d 0c 03 01
local_msgs | Apr 05 15:28:47 pethublocal/messages/D98122FEFFEC1054 606b108c 06b0 132 105 519 6 01 0c 01 0c 02 01
local_msgs | Apr 05 15:28:47 pethublocal/messages/D98122FEFFEC1054 606b108d 06c0 132 106 632 16 bc 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
local_msgs | Apr 05 15:29:11 pethublocal/messages/D98122FEFFEC1054 606b10a4 06d0 132 107 33 3 95 0c 81
local_msgs | Apr 05 15:34:00 pethublocal/messages/D98122FEFFEC1054 606b11c5 06e0 8 20 0c 05 b2 00 00 00 00 00 00 00 01 4c
local_msgs | Apr 05 15:34:00 pethublocal/messages/D98122FEFFEC1054 606b11c5 06f0 132 108 621 9 0c 05 b2 00 00 00 00 00 00
local_msgs | Apr 05 15:34:57 pethublocal/messages 606b11fe 0700 132 109 54 3 03 4f 44
local_msgs | Apr 05 15:35:02 pethublocal/messages 606b1203 0710 132 110 54 3 02 50 41
local_msgs | Apr 05 15:35:08 pethublocal/messages/D98122FEFFEC1054 606b120c 0720 8 00 0c 06 61 00 0f 01 44 01 01 01 01 5d
local_msgs | Apr 05 15:35:08 pethublocal/messages/D98122FEFFEC1054 606b120c 0730 132 111 525 3 0c 06 61
local_msgs | Apr 05 15:35:19 pethublocal/messages/D98122FEFFEC1054 606b1217 0740 8 20 0c 07 b5 00 00 00 00 00 00 02 01 5f
local_msgs | Apr 05 15:35:19 pethublocal/messages/D98122FEFFEC1054 606b1217 0750 132 112 621 9 0c 07 b5 00 00 00 00 00 00
local_msgs | Apr 05 15:38:10 pethublocal/messages 606b12c2 0760 132 113 54 1 03
local_msgs | Apr 05 15:38:10 pethublocal/messages 606b12c2 0770 132 114 56 1 45
local_msgs | Apr 05 15:38:15 pethublocal/messages/D98122FEFFEC1054 606b12c7 0780 8 01 0c 0a 61 00 02 03 44 03 03 03 01 87
local_msgs | Apr 05 15:38:15 pethublocal/messages 606b12c7 0790 132 115 54 3 02 51 42
local_msgs | Apr 05 15:38:15 pethublocal/messages/D98122FEFFEC1054 606b12c7 07a0 8 01 0c 0a 81 00 0f 04 44 04 04 04 01 88
local_msgs | Apr 05 15:38:15 pethublocal/messages/D98122FEFFEC1054 606b12c7 07b0 132 116 528 3 0c 0a 81
local_msgs | Apr 05 15:43:54 pethublocal/messages/D98122FEFFEC1054 606b141a 07c0 132 117 321 6 01 0c 01 0c 02 01
local_msgs | Apr 05 15:43:54 pethublocal/messages/D98122FEFFEC1054 606b141a 07d0 132 118 36 1 04
local_msgs | Apr 05 15:43:55 pethublocal/messages/D98122FEFFEC1054 606b141b 07e0 132 119 40 1 02
local_msgs | Apr 05 15:43:55 pethublocal/messages/D98122FEFFEC1054 606b141b 07f0 132 120 519 6 02 0c 01 0c 02 01
local_msgs | Apr 05 15:43:58 pethublocal/messages/D98122FEFFEC1054 606b141e 0800 132 121 519 6 01 0c 01 0c 02 01
local_msgs | Apr 05 15:43:58 pethublocal/messages/D98122FEFFEC1054 606b141e 0810 132 122 36 1 00
local_msgs | Apr 05 15:43:59 pethublocal/messages/D98122FEFFEC1054 606b141e 0820 132 123 40 1 02
Actually looking at it again…it thinks I have a pet feeder? But I only have a pet door and 2 pets on the door currently.
The DB looks ok though…
sqlite> select * from devices;
0000801F121D6719|1|Home hub|H004-0088731||||{"device": {"hardware": 3, "firmware": 2.43}}
D98122FEFFEC1054|3|Kitchen||4.9566|-70.16666666666667|-73.8|{"lcd": {"hardware": 1, "firmware": 1}, "rf": {"hardware": 4, "firmware": 0.16}}
sqlite> select * from doors;
D98122FEFFEC1054|0|12:01|12:02|2
sqlite> select * from feeders;
sqlite> select * from hubs;
0000801F121D6719|4|0
sqlite> select * from pets;
711033360D00|Ziva|2
Humm this shouldn’t be happening:
local_pethub | OnMessage pethublocal/messages/D98122FEFFEC1054 1 b'606b141a 07c0 132 117 321 6 01 0c 01 0c 02 01'
As OnMessage means that it isn’t calling the petdoor function, so I need to review my code this evening to see why that is happening as it should be calling on_petdoor_message but it doesn’t seem to be.
And this was just a typo where I got the door and feeder around the wrong way, it is loading it for the door.
local_pethub | Loading Pet: Ziva for <s>feeder</s> pet door Kitchen
local_pethub | Loading Pet: Jessie for <s>feeder</s> pet door Kitchen
ok great, so its working mostly.
This morning, some of the sensors started showing up. Like:
Maybe a nice improvement will be to add the following to the home assistant auto configuration:
unique_id:
An ID that uniquely identifies this sensor. If two sensors have the same unique ID, Home Assistant will raise an exception.
device:
You must have the same parameters for each entity
So for device it can be the device serial/nr as in the sqlite db
Unique id per mqtt item can be soemthing like device+sensor name.
You can see how someone else suggested this to me on something else I worked on at MQTT demultiplexer automatic sensor discovery for tasmota2zigbee (last post currently)
Then it will be much easier to find the sensors in the mqtt integration, I currently have MANY sensors in there, so difficult to figure out which ones are from local_pethub.
I also realised how I messed up but didn’t find it as my config doesn’t update from default. The web/app.py creates the topic name as pethublocal so the hub will be creating messages in pethublocal/messages whereas in the mqtt/default/conf/mosquitto.conf I had pethublocal/hub/ as I originally had pethublocal/hub/messages expecting that I would be creating topics like pethublocal/local/door
Just corrected them all locally and going to push them once I have fully tested it.
Like the suggestion of using the uniqueid as the mac/tagid. I went for the name so that human readable values were the topic names, but as you say it won’t work for everyone… let me thing about it… and patches welcome.
Ah now I know why the one mosquitto didn’t forward to the main one. I eventually added the certificates to my main mosquitto container and had everything connect directly to that.
Just added support for setting the lock state inbound and outbound on the door via the inbound and outbound switches.
Seems to work for me ™
I get this:
local_pethub | Traceback (most recent call last):
local_pethub | File "pethubmqtt.py", line 145, in <module>
local_pethub | ret=mc.publish(device_switch_t+devl+"_"+key+'/config',json.dumps(configmessage))
local_pethub | NameError: name 'key' is not defined
Do we need key here? Its not the same as the switch.
Well my pet door lock and unlock works woohoo