Jablotron JA-80 series and JA-100 series alarm integration

plaksnor · May 26, 2019, 11:13pm

@Matt, I found an issue in the _read_loop function:

2019-05-27 01:07:10 INFO (MainThread) [homeassistant.components.alarm_control_panel] Setting up alarm_control_panel.jablotron,
2019-05-27 01:07:14 INFO (ThreadPoolExecutor-0_0) [custom_components.jablotron.alarm_control_panel] ReadLoop: Acquiring lock...,
2019-05-27 01:07:14 INFO (ThreadPoolExecutor-0_0) [custom_components.jablotron.alarm_control_panel] ReadLoop: Lock acquired.,
2019-05-27 01:07:14 INFO (ThreadPoolExecutor-0_2) [custom_components.jablotron.alarm_control_panel] StartupMessage: Acquiring lock...,
2019-05-27 01:07:14 INFO (ThreadPoolExecutor-0_0) [custom_components.jablotron.alarm_control_panel] ReadLoop: opened connection,
2019-05-27 01:07:14 INFO (ThreadPoolExecutor-0_0) [custom_components.jablotron.alarm_control_panel] ReadLoop: trying to read data,
2019-05-27 01:07:14 INFO (MainThread) [homeassistant.setup] Setup of domain alarm_control_panel took 3.9 seconds.,
2019-05-27 01:07:16 WARNING (ThreadPoolExecutor-0_1) [custom_components.jablotron.alarm_control_panel] Data has not been received for 2 seconds, retry startup message,
2019-05-27 01:07:16 INFO (ThreadPoolExecutor-0_1) [custom_components.jablotron.alarm_control_panel] StartupMessage: Acquiring lock...

It’s not able to read data. Here’s a part of my code, debugging while abusing _LOGGER.info:

                self._f = open(self._file_path, 'rb', 64)
                _LOGGER.info("ReadLoop: opened connection")

                _LOGGER.info("ReadLoop: trying to read data")
                new_state = self._read()
                _LOGGER.info("ReadLoop: succesfully read data")

This is probably the reason why the Startup Message function is not able to acquire a lock (at least for my 100-series, maybe it’s not a problem for @Marcel1)

thoky · May 27, 2019, 9:33am

Hi Everybody,

I am looking at this thread for few weeks (btw very nice work ) and I have a question.
Is there is a possibility to use this integration with Raspberry even if my HA is installed on Freenas VM. Or do I need to migrate my HA to Raspberry?

Did anybody tried to use the JA-80BT Bluetooth adapter instead of the JA-82T cable?

Thanks

plaksnor · May 27, 2019, 2:45pm

Hi @thoky, thanks!

No I haven’t tried using the BT adapter, I’m sorry.

And good question. At the moment, the Jablotron plugin is part of HA, which needs to be installed on the same system which is connected directly to the alarm system. Although I see some ways to workaround this, it’s not implemented yet. For example, there’s a way to run a python script on your RPi sending/reading MQTT messages, so HA (or any other application capable of reading MQTT topics) could control your RPi remotely.

Personally, I think this is a good enhancement when we’ve made some more progress in having better control of the system. Unfortunately, we’re not at this point yet. But I like the way you’re thinking

mattsaxon · May 27, 2019, 4:20pm

@thoky, I could see way of using a remote serial port also, using something like this http://ser2net.sourceforge.net/, but as @plaksnor says, we haven’t progressed that far as yet.

mattsaxon · May 27, 2019, 4:25pm

Ah I see, for your system, the read blocks, rather than say returning an error or an empty buffer, this never happens for my system. I’ll give some consideration to how to deal with that.

plaksnor · May 27, 2019, 8:52pm

Thanks, that would be cool. I analyzed my USB recordings again and I think using another payload as a startup message would give me faster results. When I disable all acquire and release lines of code, the watcher loop is working much better and sending me something which looks like a continuous stream of data. cat /dev/hidraw0 is showing me more frequently incoming packets.
The new startup message I’m using now is:

80 01 01 52 01 0E

This will keep sending the alarm state every 2 seconds. I hope someone is able to try and confirm.

Marcel1 · May 27, 2019, 8:57pm

@plaksnor:
Regarding your question:

Are you saying you’re also getting sensor info when J/F-link is not operational?

No, not what i have seen so far. I tried to capture some data once but, but forgot to start F-link. I noticed that no data was received at all when not running the Jabltoron software on my laptop.

plaksnor · May 27, 2019, 9:03pm

Ok, this looks good. When arming the alarm, it will immediately show the orange/yellow blinking arming icon in HA, which indicates the 15 seconds entrance delay. Never happened to see this before.
So if the script is going to handle locks properly for my system, the three of us are experiencing all the same behaviour.
@Marcel1: maybe you’re able to test my new startup message as well? This would give me more info if I’m the only one who needs to use a different message.

Marcel1 · May 27, 2019, 9:12pm

@plaksnor: I just did a very quick 5 min check right now. My version was configured on 5 seconds interval with the startupmessage i mentioned earlier: \x52\x01\x02\x00\x00\x00\
I changed the code to use your new suggested startupmessage. There was no difference notable.
But to be honest, for me was already acceptable. On everage the control panel executes my commands in 5 seconds i think. Can you ellobarate a bit on the kind of delay you are facing?

Thoky81 · May 27, 2019, 9:31pm

@mattsaxon @plaksnor Thank you for reply’s I will try to get the cable and another RPi for now to see how it work. Btw, I really appreciate your work progress in this topic and wish you all the best

plaksnor · May 27, 2019, 9:41pm

Yeah sure! Ok, let’s assume we’ve no scripts running at all. Nothing is communicating with your alarm system.

Start a putty session listening to /dev/hidraw0 (or whatever device you’re using)

sudo cat /dev/hidraw0 | hexdump -C

Results: Here, nothing is happening. I don’t receive any data at all yet.

Now start a 2nd putty session and submit the default startup message I was using before:

echo -ne "\x00\x00\x01\x01" > /dev/hidraw0

Results: I do see a couple of lines, containing 51 22, which indicates the alarm is off. So far so good.

Now try to repeat step 2 every second, like you’re polling the alarm system.
In my case, nothing is happening. I only got a response from the 1st message and I don’t get any response on all other messages.
Now try to repeat step 2, but every 10 seconds. This starts to work for me, but it’s far from ideal. You don’t want to a 10 seconds delay.

So in one of my last posts I discovered that I was able to send a different message, continiously:

echo -ne "\x80\x01\x01\x52\x01\x0E" > /dev/hidraw0

Even when I am sending it multiple times a second, it’s always responding, immediately, very fast and with smaller payloads. It even seems there’s no header in the package at all.

plaksnor · May 27, 2019, 9:44pm

I’ve tried this one repeatedly, but it reacts the same way as \x00\x00\x01\x01: I still need to pause it about 10 seconds before sending the next message.

plaksnor · May 27, 2019, 10:43pm

@Marcel1: I’d happy to decipher all of your packets, but I’m still unable to find the right payload to trigger all of the \x52\x09\x8a packets. It’s good to see though that your sensor packets start with the same bytes as mine.

Marcel1 · May 28, 2019, 11:33am

@plaksnor: Totally clear now.
I experience the same as you indeed.
Only startup code \x80\x01\x01\x52\x01\x0E seems to generate a response every time, regardless the frequenty of submitting. Good stuff. How exactly did you find this code?

I just tried to modify the watcherloop and decreased the waits:

def _watcher_loop(self):

    while not self._stop.is_set():
        
        if not self._data_flowing.wait(2):
            _LOGGER.warn("Data has not been received for 2 seconds, retry startup message")
            self._startup_message()         
        else:
           _LOGGER.debug("Data is flowing, wait 1 second before checking again")
           time.sleep(1)

I haven’t seen the control panel being so responsive to HA commands before.
This is really a good improvement.

I am wondering, is there still a need for checking if data is flowing? Should’t you just send the new startup code every second?

plaksnor · May 28, 2019, 12:07pm

@Marcel1: Thanks for the confirmation. Good to know our systems respond exactly the same. The way I found this was by trial and error: I just submitted (parts of) packets which have been captured in the logfiles as sent (not received) packages. Somehow I’ve missed this one.
The other thing I noticed yesterday was the range of bytes the 100-series seems to use for sending commands to the alarm system:

\x80\x01\x01\x52\x01\x0E		startup message
\x80\x08\x03\x39\x39\x39		pre code for sending alarmcode
\x80\x02\x0d\x90				disarm
\x80\x02\x0d\xa0				arm away
\x80\x02\x0d\xb0				arm home

They all start with \x80, so this makes it easier to search.

Watcher loop:
I’m not sure if we need to check if data is flowing. I thought @Matt has no reason to repeatedly send a keepalive packet, because the 80-series seems to have a continious stream of data.
@Matt: any thoughts how we could make this more generic? I do not mind having different logic for different JA-series, but in general I’d expect both should work the same way.

JA system information:
Are you guys also seeing system information when you’re submitting this?

echo -ne "\x30\x01\x01\x30\x01\x02\x30\x01\x03\x30\x01\x04\x30\x01\x05\x30\x01\x08\x30\x01\x09\x30\x01\x0A\x30\x01\x0B\x30\x01\x0C\x30\x01\x11\x52\x03\x1A\x01\x00\x3C\x01\x01\x00" > /dev/hidraw0

The lines below represent a part of the packet I found last night, which contains additional info which might been interesting to show in HA.

30 01 01 30 01 02 30 01 03 30 01 04 30 01 05 30   0..0..0..0..0..0
01 08 30 01 09 30 01 0A 30 01 0B 30 01 0C 30 01   ..0..0..0..0..0.
11 52 03 1A 01 00 3C 01 01 00 00 00 00 00 00 00   .R....<.........
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................

It will respond with my model number

0000c640  40 0c 02 4a 41 2d 31 30  31 4b 2d 4c 41 4e 00 00  |@..JA-101K-LAN..|

My FW (firmware) version:

0000c740  40 08 08 4c 4a 36 30 34  32 32 00 10 f4 00 00 00  |@..LJ60422......|

My HW (hardware) version:

0000c780  40 08 09 4c 4a 31 36 31  32 33 00 10 f4 00 00 00  |@..LJ16123......|

And further on it will also show my Registration Code (format XXXXX-XXXXX-XXXX) and the name of my Configuration, for example “Centrale Lastname”.

That’s all for now, folks

Marcel1 · May 28, 2019, 12:32pm

Ok. Great catch. Like i said, good performance/response improvement.

The JA system information is showing here as well when sending your code:

JA-101K
LJ60422
LJ16120
…and indeed the registration code.

Maybe we’ll find a usecase later where this information can be usefull!

plaksnor · May 28, 2019, 9:57pm

I think I got some new progress.
The upgrade of the new startup/keepalive message was good, but only for getting info about the alarm state. This afternoon I thought: what if we find a packet which gives us info about sensor states, but with the same high rate frequency as the recently discovered packet?

So I analyzed things over again and applied a different keepalive packet:

echo -ne "\x80\x01\x02" > /dev/hidraw0

The system responds with 2 relevant types of packets. And for what I see, they only appear when you trigger sensors. So I extended the _read function with additional conditions:

                elif packet[:2] == b'\x55\x09' or (packet[:2] == b'\xd8\x08' and packet[10:12] == b'\x55\x09'): # sensor?
                  
                    # offset is different when packet starts with d8 08
                    if packet[:2] == b'\x55\x09':
                        sensordata = packet[0:6]
                    else:
                        sensordata = packet[10:16]

                    # get info
                    _devtyp = sensordata[2:3]
                    _state  = sensordata[3:4]
                    _device = sensordata[4:6]

                    # not sure what the 3rd byte is yet, let's try this
                    if _devtyp == b'\x00':
                        devtyp = 'magnetic or PIR'
                    elif _devtyp == b'\x01':
                        devtyp = 'PIR with photo'
                    else:
                        devtyp = 'unknown'

                    if _state == b'\x88' or _state == b'\x80':
                        sensor_state = 'opened'
                    elif _state == b'\x8a' or _state == b'\x82':
                        sensor_state = 'closed'
                    elif _state == b'\x75' or _state == b'\x79' or _state == b'\x7d':
                        sensor_state = 'triggered'
                    else:
                        sensor_state = 'unknown'

                    # most probably user specific
                    if _device == b'\x00\x02':
                        device = 'backdoor'
                    elif _device == b'\x80\x01':
                        device = 'frontdoor'
                    elif _device == b'\x40\x01':
                        device = 'studio'
                    elif _device == b'\xc0\x00':
                        device = 'hall'
                    elif _device == b'\x00\x01':
                        device = 'garage'
                    else:
                        device = 'unknown'

                    _LOGGER.info("Sensor changed: packet info: %s %s %s", _devtyp, _state, _device)
                    _LOGGER.info("Sensor changed: resolves to: %s %s %s", devtyp, sensor_state, device)
                    pass

Is someone able to test this by using the new startup message and applying the above code as extended condition in the _read function? I’ll change my github repo later on when I’ve done some more tests.

mattsaxon · May 29, 2019, 8:15am

To confirm, the JA-80 doesn’t need the data flowing tests, that was added for the JA-100 series control panels, which I don’t have access to.

mattsaxon · May 29, 2019, 8:16am

Given all the great progress you’ve made on the JA-100 series, shall we take this opportunity to rename the repo to remove the 80 designation. I wanted to check as it might require you to reclone the repo (though somethings in GitHub are resilient to such name changes, last time I check clone repos weren’t)

plaksnor · May 29, 2019, 8:50am

Sure, great suggestion!
Another suggestion for new development: could we make it as a multiple platform? At the moment, it’s an alarm_control_panel platform, but I’d like to extend it with a sensor platform as it looks like we’re also getting sensor data.

Ps. I noticed I was using ‘state’ as variable in my last post, but that variable was already being used and could mess up your alarm state