WTH can remote serial devices not be easily mounted as /dev/tty virtual devices?

ryans · September 21, 2020, 8:06am

@infocus13 is correct! This is NOT specific to the Serial integration. The goal would be to to allow ALL integrations that rely on directly reading/writing /dev/tty* devices to be able to interact with remote serial devices. Basically, every integration that interacts over RS232/RS485/etc should be able to easily communicate with the device whether it is locally attached OR remotely attached via a common serial-over-IP protocol. socat is one way to achieve this (and probably the simplest to leverage) as it is commonly used for this.

For example, the following core integrations are impossible to use with a remote tty natively from Home Assistant, without installing socat package and hand configuring mounts:

Similarly, there are many custom components that also communicate via serial communication…ALL of these would benefit. This is especially useful for many home audio receivers/amplifiers, which often provide RS232 control, but no network accessible API.

With addition of socat and some minor UI addition (or yaml configuration), ALL integrations would be able to remotely interact with serial devices connected over IP.

darrylb · December 13, 2020, 12:08am

Did anything come from this? I’m thinking of virtualize going for redundancy after continual NUC failures and this will be a hurdle.

ryans · December 16, 2020, 6:44am

@darrylb I don’t think anyone is working on this yet, but hopefully someone picks this up and runs with it!

beerygaz · January 10, 2021, 9:33am

I am kinda gobsmacked this is not already available. I have many serial devices that are only accessible over GlobalCache or other ser2net gateways.

Given I’m new to HA I’d love to stick with the simplicity of hassos - I’m guessing right now the next best option is to build my own container stack? I’m worried about the future upgrade burden this puts on the system as I may drift further from the “supported” platform.

aceindy · January 10, 2021, 10:07am

I was under the impression that was already working, f.e. with zigbee i can choose to use serial or tcp

Just wondering how this related to this topic

ryans · March 15, 2021, 6:45am

This is not supported in HA as a generic way to mount remote serial ports (e.g. using socat or other). EACH integration has had to separately implement serial control, if they want to support it. For instance, the Zigbee integration you pointed had to manually implemented communicating via serial.

doolio · August 9, 2021, 4:54pm

I would love to see this added too and honestly with the number of Serial/RS-232 devices that are out there I’m really surprised that it hasn’t been added to HA yet. I have one old AV Matrix that is the last item in my setup that I’d like to integrate into HA but I can’t do it since I’m not a SW developer. If anyone has any updates on how to integrate Serial devices then please share

thomas-be · September 1, 2021, 10:07am

Still waiting for this feature too. It would be great to see a little more “abstraction” between HA and the hardware it is connected to. Allowing people to connect to any serial device over network seems like a basic thing to implement for home automation software.

ryans · September 1, 2021, 3:10pm

What AV matrix is that you have?

tinglis1 · September 2, 2021, 12:00am

I would like something built into HA for tcp serial. I wanted to share my workaround for a situation I have.

I have a Anthem MRX500 receiver that has rs232 only for control/monitoriong.
I have a IP-to-rs232 device (global cache iTach) connected to the receiver.
Node Red to communicate with a IP-to-rs232 device.
Send events that NR monitors and acts on.
NR monitors the tcp connection and parses the responses.
Returned data is sent as events to HA.
Automations/sensors etc in HA use the event data.

It still needs some polishing but the concept works for me and might for some of your situations.

Here is the NR flow for those who want it.

[{"id":"c9e6e970e7729a45","type":"tab","label":"mrx","disabled":false,"info":""},{"id":"8132adaa70942957","type":"tcp request","z":"c9e6e970e7729a45","server":"192.168.20.134","port":"4999","out":"char","splitc":"0xa","name":"","x":390,"y":540,"wires":[["14b2d28e8de2f5d2"]]},{"id":"14b2d28e8de2f5d2","type":"function","z":"c9e6e970e7729a45","name":"","func":"msg.payload = msg.payload.toString('utf8');\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":620,"y":540,"wires":[["5792ecefde09dc38","514d70cdd8bb0d38"]]},{"id":"ffc4835302c239bf","type":"server-events","z":"c9e6e970e7729a45","name":"","server":"f41510c7.a2386","version":1,"event_type":"anthem_mrx","exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"waitForRunning":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"eventData"},{"property":"topic","propertyType":"msg","value":"$outputData(\"eventData\").event_type","valueType":"jsonata"}],"x":110,"y":320,"wires":[["ecc5ab4c30f7ad1e","1ad8f3cc011dc350"]]},{"id":"6b4049cbc503658b","type":"function","z":"c9e6e970e7729a45","name":"add \\x0a","func":"msg.payload = msg.payload + \"\\x0a\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":800,"y":320,"wires":[["18543f9ffde65ec6","e100c19bbae4a84c"]]},{"id":"18543f9ffde65ec6","type":"debug","z":"c9e6e970e7729a45","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":970,"y":260,"wires":[]},{"id":"ecc5ab4c30f7ad1e","type":"debug","z":"c9e6e970e7729a45","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":310,"y":380,"wires":[]},{"id":"b29887a1a73587b1","type":"ha-fire-event","z":"c9e6e970e7729a45","name":"","server":"f41510c7.a2386","version":0,"event":"anthem_mrx","data":"","dataType":"json","x":750,"y":800,"wires":[["8b5e354855a89558"]]},{"id":"1ad8f3cc011dc350","type":"switch","z":"c9e6e970e7729a45","name":"only send events","property":"payload.event.send","propertyType":"msg","rules":[{"t":"nempty"}],"checkall":"true","repair":false,"outputs":1,"x":350,"y":320,"wires":[["0d73c4de97520ccd"]]},{"id":"5792ecefde09dc38","type":"debug","z":"c9e6e970e7729a45","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":790,"y":480,"wires":[]},{"id":"fd84c80ae0667a44","type":"inject","z":"c9e6e970e7729a45","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"P1?","payloadType":"str","x":110,"y":140,"wires":[["b17c136b939ab089"]]},{"id":"4a076656ef5765d0","type":"inject","z":"c9e6e970e7729a45","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"P1V-59","payloadType":"str","x":110,"y":180,"wires":[["b17c136b939ab089"]]},{"id":"24e22ee739fe9475","type":"inject","z":"c9e6e970e7729a45","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"P1P0","payloadType":"str","x":110,"y":100,"wires":[["b17c136b939ab089"]]},{"id":"ee91995f75cab87c","type":"inject","z":"c9e6e970e7729a45","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"P1P1","payloadType":"str","x":250,"y":100,"wires":[["b17c136b939ab089"]]},{"id":"0d73c4de97520ccd","type":"change","z":"c9e6e970e7729a45","name":"move event send","rules":[{"t":"move","p":"msg.payload.event.send","pt":"msg","to":"msg.payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":320,"wires":[["6b4049cbc503658b","47315587052f1d6d"]]},{"id":"d5bb86918d28d34f","type":"switch","z":"c9e6e970e7729a45","name":"query","property":"payload","propertyType":"msg","rules":[{"t":"cont","v":"?","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":190,"y":1060,"wires":[[],["076483599ad55868"]],"outputLabels":["yes","no"]},{"id":"96716abf63261ca6","type":"change","z":"c9e6e970e7729a45","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"P1?","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":580,"y":1060,"wires":[["1352090141652f40","1fe114fc39c75a2a"]]},{"id":"076483599ad55868","type":"delay","z":"c9e6e970e7729a45","name":"","pauseType":"delay","timeout":"500","timeoutUnits":"milliseconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"x":370,"y":1060,"wires":[["96716abf63261ca6"]]},{"id":"b17c136b939ab089","type":"link out","z":"c9e6e970e7729a45","name":"","links":["ecc3427a301c15ad"],"x":395,"y":100,"wires":[]},{"id":"d338be1d8a537133","type":"link in","z":"c9e6e970e7729a45","name":"","links":["1352090141652f40"],"x":615,"y":260,"wires":[["6b4049cbc503658b"]]},{"id":"e2cb8a287389d028","type":"link in","z":"c9e6e970e7729a45","name":"","links":["47315587052f1d6d"],"x":75,"y":1060,"wires":[["d5bb86918d28d34f"]]},{"id":"e100c19bbae4a84c","type":"link out","z":"c9e6e970e7729a45","name":"","links":["db4f35fa89e37868"],"x":935,"y":320,"wires":[]},{"id":"1352090141652f40","type":"link out","z":"c9e6e970e7729a45","name":"","links":["d338be1d8a537133"],"x":855,"y":1060,"wires":[]},{"id":"1073b6c8dfbe31f7","type":"comment","z":"c9e6e970e7729a45","name":"If the send message is a command send a follow up query","info":"","x":270,"y":1000,"wires":[]},{"id":"e305fb8a6645e8a2","type":"comment","z":"c9e6e970e7729a45","name":"send the mesage and get the response","info":"","x":190,"y":480,"wires":[]},{"id":"20776412396a6c82","type":"comment","z":"c9e6e970e7729a45","name":"test commands","info":"","x":120,"y":40,"wires":[]},{"id":"4d0487310ad158c0","type":"comment","z":"c9e6e970e7729a45","name":"reeived event and create tcp message","info":"","x":170,"y":280,"wires":[]},{"id":"db4f35fa89e37868","type":"link in","z":"c9e6e970e7729a45","name":"","links":["e100c19bbae4a84c"],"x":55,"y":540,"wires":[["295ae929b968e149"]]},{"id":"47315587052f1d6d","type":"link out","z":"c9e6e970e7729a45","name":"","links":["e2cb8a287389d028"],"x":755,"y":380,"wires":[]},{"id":"1fe114fc39c75a2a","type":"debug","z":"c9e6e970e7729a45","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":850,"y":1120,"wires":[]},{"id":"d1f939d0a1ea6e1f","type":"comment","z":"c9e6e970e7729a45","name":"parse the repsponse","info":"","x":130,"y":700,"wires":[]},{"id":"bf9d9e038d9be7cb","type":"link in","z":"c9e6e970e7729a45","name":"","links":["18068d9d9e112f6b"],"x":55,"y":800,"wires":[["d9dc94f16fd1410f"]]},{"id":"18068d9d9e112f6b","type":"link out","z":"c9e6e970e7729a45","name":"","links":["bf9d9e038d9be7cb"],"x":1055,"y":540,"wires":[]},{"id":"514d70cdd8bb0d38","type":"switch","z":"c9e6e970e7729a45","name":"response contains zone flag","property":"payload","propertyType":"msg","rules":[{"t":"nnull"}],"checkall":"true","repair":false,"outputs":1,"x":860,"y":540,"wires":[["18068d9d9e112f6b"]]},{"id":"875b1e7e0c7f2186","type":"inject","z":"c9e6e970e7729a45","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"P1V?","payloadType":"str","x":110,"y":220,"wires":[["b17c136b939ab089"]]},{"id":"d9dc94f16fd1410f","type":"switch","z":"c9e6e970e7729a45","name":"","property":"payload","propertyType":"msg","rules":[{"t":"regex","v":"P(.)S(.)V(.*?)M(.)D(.)","vt":"str","case":false},{"t":"regex","v":"P(.)VM(.*)","vt":"str","case":false},{"t":"cont","v":"Main Off","vt":"str"},{"t":"cont","v":"Zone2 Off","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":5,"x":150,"y":800,"wires":[["2f7a889f64e96de6"],["c01d41cab5e1715b"],["0f96929e6e115670"],["e4f47f6b164900ee"],["d9f7cb464650bf40"]],"outputLabels":["Pz?","","","",""]},{"id":"c01d41cab5e1715b","type":"function","z":"c9e6e970e7729a45","name":"PzV?","func":"var regExp = /P(.)VM(.*)/;\nvar results = regExp.exec(msg.payload);\n\n// organise the zone data\nrecv = {};\nrecv.raw = msg.payload;\n\nzone = {};\nzone.power = 1;\nzone.volume = results[3];\n\nrecv[results[1]] = zone;\n\n// creat the payload for the HA event\nmsg.payload = {};\nmsg.payload.data = {};\nmsg.payload.data.recv = recv;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":310,"y":800,"wires":[["b29887a1a73587b1"]]},{"id":"40654b7215955d45","type":"inject","z":"c9e6e970e7729a45","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"P1MT","payloadType":"str","x":250,"y":140,"wires":[["b17c136b939ab089"]]},{"id":"ecc3427a301c15ad","type":"link in","z":"c9e6e970e7729a45","name":"","links":["b17c136b939ab089"],"x":615,"y":220,"wires":[["6b4049cbc503658b","47315587052f1d6d"]]},{"id":"8fb06ee60c7189a2","type":"inject","z":"c9e6e970e7729a45","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"P2?","payloadType":"str","x":250,"y":180,"wires":[["b17c136b939ab089"]]},{"id":"295ae929b968e149","type":"delay","z":"c9e6e970e7729a45","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"4","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"allowrate":false,"x":170,"y":540,"wires":[["8132adaa70942957"]]},{"id":"2f7a889f64e96de6","type":"function","z":"c9e6e970e7729a45","name":"Pz?","func":"var regExp = /P(.)S(.)V(.*?)M(.)D(.)/;\nvar results = regExp.exec(msg.payload);\n\n// organise the zone data\nrecv = {};\nrecv.raw = msg.payload;\n\nzone = {};\nzone.power = 1;\nzone.source = results[2];\nzone.volume = results[3];\nzone.mute = results[4];\nzone.decoder = results[5];\n\nrecv[results[1]] = zone;\n\n// creat the payload for the HA event\nmsg.payload = {};\nmsg.payload.data = {};\nmsg.payload.data.recv = recv;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":310,"y":760,"wires":[["b29887a1a73587b1"]]},{"id":"8b5e354855a89558","type":"debug","z":"c9e6e970e7729a45","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":930,"y":800,"wires":[]},{"id":"d9f7cb464650bf40","type":"function","z":"c9e6e970e7729a45","name":"raw only","func":"// organise the zone data\nrecv = {};\nrecv.raw = msg.payload;\n\n// create the payload for the HA event\nmsg.payload = {};\nmsg.payload.data = {};\nmsg.payload.data.recv = recv;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":320,"y":920,"wires":[["b29887a1a73587b1"]]},{"id":"0f96929e6e115670","type":"function","z":"c9e6e970e7729a45","name":"P1P0","func":"// organise the zone data\nrecv = {};\nrecv.raw = msg.payload;\n\nzone = {};\nzone.power = 0;\n\nrecv[1] = zone;\n\n// creat the payload for the HA event\nmsg.payload = {};\nmsg.payload.data = {};\nmsg.payload.data.recv = recv;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":310,"y":840,"wires":[["b29887a1a73587b1"]]},{"id":"e4f47f6b164900ee","type":"function","z":"c9e6e970e7729a45","name":"P2P0","func":"// organise the zone data\nrecv = {};\nrecv.raw = msg.payload;\n\nzone = {};\nzone.power = 0;\n\nrecv[2] = zone;\n\n// creat the payload for the HA event\nmsg.payload = {};\nmsg.payload.data = {};\nmsg.payload.data.recv = recv;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":310,"y":880,"wires":[["b29887a1a73587b1"]]},{"id":"f41510c7.a2386","type":"server","name":"Home Assistant","version":1,"legacy":false,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true}]

doolio · September 2, 2021, 5:50am

Here is the solution I came up with to control my Elan S12R AV matrix switch:

I have it hooked up to a Global Cache IP2SL and using their utility iTest I realized that I could successful communicate to my S12 through my IP2SL when I was connected with Telnet.

Lightning struck and I realized that HA has a Telnet service. From there it literally took me minutes to write some code to send serial commands with the Telnet service to the IP2SL/Elan. I have it all set up and running perfectly. I even exposed it to Alexa and can now control it with voice commands.

aceindy · September 2, 2021, 11:00am

This basically means we could connect any device with JeeLabs ESPlink over ethernet, as it also support Telnet
For sure cheaper thenIP2SL

Any details on how you send serial over telnet ?

doolio · September 7, 2021, 5:30am

Here is the documentation for the Telnet service. It’s pretty straight forward. I took the serial commands for my AV Matrix and entered them in and it worked immediately (again through the Global Cache IP2SL gateway).

ryans · September 7, 2021, 7:25am

The problem is that you can’t use the Telnet service with any of the existing Home Assistant integrations that communicate RS232 over /dev/tty* local connections. That is the main issue here…just mapping Telnet into custom YAML doesn’t solve this since you have to replicate basically the entire integration.

What is needed is to be able to create a local mount point that proxies to the remote RS232 connection. Using something like iocat.

doolio · September 7, 2021, 7:33am

Yes I totally agree. This is simply a workaround and still doesn’t address all of the use cases. For example, since it’s a switch I’m not sure if I can use it to adjust the volume. Since it’s a Matrix I had to write a ton of repetitive yaml to make it work. But at least it got the AV Matrix (which was the last key integration) into HA. Something is better than nothing and I thought it could help someone else.

SeldomSeen31 · September 23, 2021, 4:35pm

Are you sending HEX commands? I’ve been struggling to set up my Sonance Whole Home Audio 12 channel amp with home assistant for a bit and just haven’t been able to get anything to work. @ryans has helped me with a few attempts but either due to my novice abilities with HA or the quirkiness of global cache and the sonance amp, nothing has worked. This is a main piece of my home. We use it every day and control it with ip2sl.

I can send HEX with the itest, zone one power on command is:
3a 5a 31 31 0D

But I have not been able to figure out the configuration to get this working in HA.

SeldomSeen31 · September 23, 2021, 5:34pm

Could you provide a bit of your yaml set up for this? I’m really struggling to implement something similar for a 12 channel amp I use for music to 6 zones in my home. I currently control it with ip2sl and another bit of software that runs on my phone. I’d like to move to HA for everything and that seems doable for all of my other equipment and devices, but this piece has put a stop to that, as if this doesn’t work then there isn’t much point in the rest. It’s too expensive to replace just because I can’t control it how I want.

jpanda · October 26, 2021, 7:54am

Just to add another voice to thread - I’m looking to integrate HiFi devices that present a IP/Serial programming interface, but I not seeing an obvious way to leverage these in HA. The current focus of my attention is an Arcam SA30, but I have other kit I’d like to control in this way.

I’m no dev, but it looks as though I going to need to get hands dirty

SeldomSeen31 · October 28, 2021, 7:32pm

I’m using a global cache ip2sl device to control my amp. I finally got it working in a not very elegant way, but don’t really care as it works as I need it. The telnet switch was what worked for me. I just added a switch for each command. If you can control your device with IP commands, this might work for you too.

switch:
  - platform: telnet
    switches:
        wha_living_room_on:
          resource: 192.168.1.114
          port: 4999
          command_on: "':Z11\x0D'"
          command_off: "':Z11\x0D'"
          name: "LR On"
          timeout: 3

        wha_living_room_off:
          resource: 192.168.1.114
          port: 4999
          command_on: "':Z10\x0D'"
          command_off: "':Z10\x0D'"
          name: "LR Off"
          timeout: 3

        wha_living_room_up:
          resource: 192.168.1.114
          port: 4999
          command_on: "':V1++\x0D'"
          command_off: "':V1++\x0D'"
          name: "WHA Living Room"
          timeout: 3 
          
        wha_living_room_down:
          resource: 192.168.1.114
          port: 4999
          command_on: "':V1--\x0D'"
          command_off: "':V1--\x0D'"
          name: "WHA Living Room"
          timeout: 3

ryans · October 28, 2021, 9:39pm

The point of being able to mount remote serial devices (like IP2SL) to a /dev/xxxx mount point is that you could use EXISTING integrations that use serial out of the box WITHOUT having to do anything special except point to the /dev/xxx mount point (once it is mapped to an IP). No knowledge required of the actual serial protocol required since the existing integration would just work.