Add RFLink Sensor/Switch 'value_template' attribute

@aequitas

I am trying to get data from my MiLight remote control into HASS via your RFLink component but I can’t find a way to get all the info I need from the packet. The ‘decoded packet’ event contains everything I want, here is an example:

2017-05-15 19:57:58 DEBUG (MainThread) [rflink.protocol] decoded packet: {'switch': '02', 'node': 'gateway', 'id': '3d47', 'protocol': 'milightv1', 'rgbw': '8060', 'command': 'on'}

This has the button which was pressed on the remote (‘switch’: ‘02’), the brightness and colour requested (‘rgbw’: ‘8060’) and the command (‘on’ or ‘off’ or MODE1, MODE2, … MODE9). What do I need to do to expose this data? also, currently pressing the MODE button on the remote causes RFLink to reject the packet:

2017-05-15 20:01:54 DEBUG (MainThread) [rflink.protocol] received data: 20;89;MiLightv1;ID=3D47;SWITCH=00;RGBW=8060;CMD=MODE2
2017-05-15 20:01:54 WARNING (MainThread) [rflink.protocol] dropping invalid data: 20;89;MiLightv1;ID=3D47;SWITCH=00;RGBW=8060;CMD=MODE2

why is that?

I have tried using MQTT sensors configured to pull json values from MQTT events using the MQTT components’s ‘value_template’ attribute which works but is WAY too slow and the MQTT client firmware running on my NodeMCU connected to my RFLink Gateway is unreliable.

My config.yaml entry for MQTT sensors with value templates:

sensor:
    # Milight remote controller 3D47 monitor
  - platform: mqtt
    state_topic: 'RF/MiLightv1-ID=3D47'
    name: 'Raw 3D47'
    value_template: '{{ value_json.raw }}'
  - platform: mqtt
    state_topic: 'RF/MiLightv1-ID=3D47'
    name: 'Button 3D47'
    value_template: '{{ value_json.SWITCH }}'
  - platform: mqtt
    state_topic: 'RF/MiLightv1-ID=3D47'
    name: 'RGBW 3D47'
    value_template: '{{ value_json.RGBW }}'
  - platform: mqtt
    state_topic: 'RF/MiLightv1-ID=3D47'
    name: 'Command 3D47'
    value_template: '{{ value_json.CMD }}'

If RFLink Sensor (or switch?) supported value_templates I would be sooooooooo happy I think I’d burst. I also think that a value_template attribute would be an easy way to accommodate the vast amount of ways that all the different protocols supported by RFLink transmit their data.

Sorry for the long post.

Currently rflink component does not support milight. I plan to start working on it some time soon. If you subscribe to this issue on HA github you will get notified when there is an implementation to test: https://github.com/home-assistant/home-assistant/issues/6851

Since rflink uses a pretty standard protocols for all devices. So I don’t know if a value template would be required. But it might be there are some use cases I have not foreseen so it might be interesting to investigate.

Could you give an example of how in your setup with your devices you would like to configure the rflink component using value_template and give a detailed description of how your device operates (eg: i press button X, rflink protocol sends command MODE2, I expect rflink component in HA to perform action Y).

This way I can understand the use case and try to come up with a solution that is most flexibel and easy.

Short answer:

If I could enter each milight controller attribute as seperate sensors just like my MQTT sensors example above I would be a very happy bunny. I could do everything else I need through yaml.

I’m no programmer but I don’t think there is any need for your component to do everything the LimitlessLED component currently does in controlling the lights.

Some users may just want to use RFLink componment to notify HASS of a change in state of the lights to update the frontend but I would like to have HASS intercept the commands and perform some logic before passing an action onto the lights.

I need the sequence number, contollrer ID, RGBW value, switch number and command from the received packet to do everything I want. Maybe I should learn python and use Appdaemon to process the incoming packet event.

Long answer:

Currently I have 11 different MiLight bulbs/strips around my home. I can control therm from HASS or various remote controls scattered about my home. The problem is that due to the MiLight protocol being fire and forget with no feedback received from the devices using the MiLight remote controls to change the state of the lamps does not update the HASS state machine.

One problem I have encountered is with motion activated lights. I have a wireless PIR in my hallway and if i walk infront of it the hallway light comes on. After 2 minutes the light goes off again. Great, but if I am doing something in the hallway and switch the light on with the wall mounted MiLight controller I can not avoid triggering the PIR which will force my lights to a different setting according to the automation and then turn the light off after two minutes when I’m trying to find something in a very untidy cupboard.

In an ideal world I would like to to be able to receive and store the values received from the remotes via RFLink and have those values trigger some sequence/event that actions the LimitlessLED component to actually change the states of the lamps accordingly. I could even use the last known state of the remote as a condition within automations! So the lights are not paired to the lights only HASS is paired to the lights.
I would also have input selects for each light so I could pick and choose which lights listen to which remotes on the fly from the frontend. No more messing about with pairing/unpairing.

Currently none of the open source MiLight controller software is able to use the built-in MiLight modes to trigger the bulbs into any of the disco modes but I thought if I could at least see that the mode button had been pressed I could use that as some sort of special case. Maybe if HASS knows the mode button was pressed it would listen for the next four presses and see if they match a code number to disable the intruder alarm or launch the nukes etc instead of operating the lights as normal. You could even write your own disco modes!

The MiLight packet contains a sequence number which would be needed to know in what order the buttons have been pressed. This is import for seeing if a button has been held down. If I hold down a zone ‘on’ button the same message is received but each meassage has an incremented sequence number. Once this counter has incremented three times the lights enter white mode, if I hold ‘off’ they go into night mode. If I want to disarm my alarm I would want to know the sequence of button presses matched an expected value.

The decoded packet contains everything needed to implement this except the sequencenumber:

2017-05-16 20:29:31 DEBUG (MainThread) [rflink.protocol] decoded packet: {'command': 'on', 'switch': '00', 'protocol': 'milightv1', 'node': 'gateway', 'id': '3d47', 'rgbw': '00e0'}

Command: on/off/color/mode0/mode1/mode2/mode3/mode4/mode5/mode6/mode7/mode8/disco-/disco+

mode: each time mode is pressed it steps the light onto the next disco mode (random colours/flashing
colours/patterns). there is only one mode button but the protocol seems atttach a counter to it.

color: sent when touching the colour wheel

disco-: slows the disco mode

disco+: speeds up the disco mode

Switch: 00/01/02/03/04
00: all groups
01: group one
02: group two
03: group three
04: group four
switch number received lets you know which group of lights to turn on or off and also which group to send any following changes in colour or brightness or mode to.

RGBW: four hex values
first two hex values = colour (red=00, blue=BE, green=86, yellow=3E, back round to red=FF)
second two hex values= brightness (00=dim, FF= brightest), anything under 06 is night mode

ID: unique to each remote control

Every open source impletation of MiLight control I have seen has been based on the brilliant Openmili project by HenryK maybe this may be some use to you (I believe this is what the Home Assistant LimitlessLED component is based on):

I realise you have your work cut out in keeping RFLink upto date so maybe adding some templating features would allow users to make use of the data without waiting for new devices to be supported everytime. I really think that when the MQTT component added templating it made the platform much more powerful.

I’m really not a programmer so I may not have included any actually useful information for you in this post but hopefully you get an idea of what I would like to do and perhaps you will be able to ask me for some more applicable data.

I really appreciate everything you’ve done to implement RFLink support in HASS.

EDIT: I thought I’d also mention another great project that I am using which emulates MiLight hubs on a NodeMCU by Christopher Mullins:

http://blog.christophermullins.com/2017/02/11/milight-wifi-gateway-emulator-on-an-esp8266/

His project has a very cool web UI which may be of some interest to someone who knows what they are looking at!

Thats quiet a read :). I’ll look into it later today if possible. For completeness could you try to create a HASS configuration of how you think you would like to configure all the features you want. So I get an idea which components (switch/sensor/light) you expect and what unique properties are to be configured for each.

This is what the milight part of my HomeAssistant config looks like for esp8266_milight_hub using MQTT:

- name: "Office Lamp"
  command_topic: milight/0/rgb_cct/1
  <<: &MILIGHT_PARAMS
    platform: mqtt_json
    color_temp: true
    rgb: true
    brightness: true
    optimistic: false
- name: "Livingroom Lamp"
  command_topic: milight/0/rgb_cct/2
  <<: *MILIGHT_PARAMS
- name: "Bedroom Lamp"
  command_topic: milight/0/rgb_cct/3
  <<: *MILIGHT_PARAMS
- name: "Nightstand Lamps"
  command_topic: milight/1/rgbw/0
  <<: *MILIGHT_PARAMS

@Whiskey @sidoh I think I get the global gist of the milight command. I don’t have any hardware of this type so it is hard for me to develop on it. A template solution might work here but I don’t want to put complexity into the rflink module if it is not needed. And this functionality may already be provided by HA in a different way.

Have you tried looking at the events that are fired by the rflink component? https://home-assistant.io/components/switch.rflink/ (fire_event option). I don’t think it will support the full range of features as I think it will only work in the context of automations but it might already provide some functionality.

Just trying to find clarification on this… Does HASS let you discover milight bulbs with RFLink, as I am really struggling to discover mine :unamused:

@aequitas I’m not really familiar with rflink. Just wanted to show that supporting the mqtt_json component makes integrating lights with HASS super easy.

Also a heads up that the newer Milight bulbs respond to a different protocol that employs some really obnoxious scrambling. It took quite a bit of effort, but I reversed it and documented it here:

http://blog.christophermullins.com/2017/03/18/reverse-engineering-the-new-milightlimitlessled-2-4-ghz-protocol/

@sidoh, I would love a bit more info on how you have used MQTT and MiLight. I found MQTT quite slow and I could also only receive a message when my finger released a button and not a continuous stream like Jonah’s RFLink component does, for example when I slide my finger around the remote’s colour wheel MQTT only fires the colour I was on when my finger releases the remote, RFLink spews out events rapidly when any button is held down. I agree that MQTT makes things super easy but I was disappointed with the latency and also the NodeMCU running an MQTT client crashed regularly. You also make a very valid point about the complecated nature of the latest MiLight protocol (which I believe is version 6, don’t quote me on that) and that was one of my reasons for using the existing LimitlessLED HASS component which already does a fantastic job at controlling my RGBWW and RGBW milight bulbs.

EDIT: It seems RFLink firmware does not currently support RGBWW protocol! I only own a RGBW remote and can see key presses in the RFLink log no problem but when I send commands to my RGBWW lights I do not see any packets!!! My RGBWW lights are only controlled from HASS so the state of those lights are always known. The rest of the house is RGBW and controlled from various points outside of HASS so adding the following functionality would be a huge help in my situation.

I have been thinking that I would take the messages recived by RFLink from the MiLight controllers and pass them to my lights declared in my config under the LimitlessLED platform. I now think it would be a more complete solution for my lights to be defined under the RFLink platform as the received controller messages can be parsed to the lights with little manipulation. The controller messages would have to have the hex values for colour and brightness converted to decimal values between 0 and 255 for each colour component and brightness. This is obviously possible and is how I was planning on doing things but after thinking about it I feel RFLink would be most complete if it could action the lights directly. Although the best way to implement this is reliant upon the RFLink gateway firmware as MiLight support has only just been implemented and may still be limited in functionality.

EDIT: Actually if the messages from the remotes could be converted into decimal values then ANY light could be configured to respond to the MiLight remote not just MiLight bulbs, that would be super cool!

@noodlemctwoodle, I don’t think you will be able to discover the lights as the MiLight protocol is one way only, i.e. the lights do not transmit any data. The remotes/hubs fire instruction to the lights and presume the lights changed state accordingly. Only these instructions can be intercepted by RFLink.

So, putting that altogether maybe my config would look like this:

light:
  platform: rflink
  devices:
    milight:
      lounge_lights: # unique name for the milight bulb
        name: Lounge # friendly name
        type: rgbw # do we need to know this to form the command properly?
        paired_id: 
          - 7E12 # ID to use when sending commands to the light / the ID of the 
                 # hub that the light is actually paired to.
            group: 1 # which group light is paired to so we know which 'SWITCH' number to use
          - 2A34 # maybe the light is paired to multiple controllers
            group: 4
          # packets recieved with any of the listed 'paired_id' with a matching group number
          # will only update the state of the light accordingly and not send another packet
      hallway_lights:
        name: Hallway
        type: white
        paired_id: 7E12 # hallway paired to one of the same hubs as lounge but on a different group
        group: 2

  platform: generic_light # add some other type of light and maybe the MiLight controller can operate that too!
    name: dungeon 
    
# is it possible to declare remote controls like this and have all the required data attributes stored 
# automatically just from defining the 'type' of controller?
sensor:
  platform: rflink
  devices:
    millight_controller:
      lounge_wall_controller:
        name: Lounge controller
        id: 3D47 # ID of remote found from HASS log of RFLink events
        type: RGBW
      hallway_remote:
        name: Hallway controller
        id: 8F19 

# or does each controller need each attribute defined just like the MQTT example?
sensor:
  platform: rflink
  device:
    milight_controller:
      lounge_controller_rgb:
        id: 3D47 # the rgb value from event with this ID will be stripped from the packet and stored in this sensor
        template_value: rgbw # need to strip the third and fourth hex values from the four hex value
      lounge_controller_switch:
        id: 3D47
        template_value: switch
      lounge_controller_brightness:
        id: 3D47
        template_value: rgbw # need to strip the first and second value from four hex value
      lounge_controller_sequence:
        id: 3D47 # if three identical packets received with concurrent sequence numbers triggers a 'long press'
        template_value: # sequence number follows the '20' at the start of the packet
      lounge_controller_command:
        id: 3D47 # on/off/mode/speed up/speed down

input_boolean:
  lounge_lights_3D37: # if on controller 3D37 will operate lounge lights
    name: lounge lights
  hallway_lights_3D37: # if on controller 3D37 will operate hallway lights
    name: hallway lights
  lounge_lights_8F19: # if on controller 8F19 will operate lounge lights
    name: lounge lights
  hallway_lights_8F19: # if on controller 8F19 will operate hallway lights
    name: hallway lights

groups:
  controller_3D47:
    name: Lounge MiLight Controller Pairing
    entities:
      - input_boolean.lounge_lights_3D47
      - input_boolean.hallway_lights_3D47
  controller_8F19:
    name Hallway MiLight Controller Pairing
    entities:
      - input_boolean.lounge_lights_8F19
      - input_boolean.hallway_lights_8F19

automation:
  alias: packet received from lounge controller for hallway lights
  trigger:
    platform: event
    event_type: rflink_packet_recieved
    event_data:
     id: 3D47
  condition: state
    entity_id: input_boolean.hallway_lights_3D47
    state: 'on'
  action:
    service: light.turn_on
      entity_id: light.hallway_lights
      data:
        rgb_color: [{{trigger.data.red}},{{trigger.data.green}},{{trigger.data.blue}}]
        brightness: {{trigger.data.brightness}}
        # I realise my action template won't work but hopefully you get the idea

I think I might have got in a bit over my head as I really don’t know what I’m doing but maybe this post has enough nonsense in it for you guys to try and make some magic happen!!

In this setup I have only paired each light to a hub and not to any remote controllers. I like the idea that I can use HASS to choose which remote triggers which light but some users may want to pair the lights to a MiLight controller and have HASS just update the state of the light without sending any actions to the light. I have included an ‘update_only’ attribute to choose whether a message received from a particular ID should cause the component to pass on the packet to the ‘paired_id’ or not. I’m not sure if anyone would find this useful or not. Packets received from the ‘paired_id’ should obviously only update the state of the light.

I’m sorry this post turned into a bit of a mess but the more I thought about it the more I confused myself. Maybe I don’t need to define the controllers and can just write automations to deal with RFLink events. Thats what I was going to do originally but I couldn’t get the sequence number from the event log so couldn’t detect a long press.

I just want my MiLight controllers to update the state of the lights and tell HASS the lights have changed state!

To be clear, I’m not using rflink. What I pasted is the HASS config I use to integrate with my Milight-specific project, esp8266_milight_hub. Just wanted to demonstrate that supporting the JSON schema used by mqtt_json makes it super easy to integrate with HASS.

If implemented properly, MQTT is actually quite a bit more responsive than something like HTTP/REST because everything is happening over already-opened TCP connections. If the publisher is opening a new connection each time it’s publishing a message, it would be about the same. My lights respond pretty much instantaneously (probably within 100ms), even when triggered with something like a zwave remote.

I understand. I have tried MQTT to control my MiLights and it was extremely easy to setup. The problem I had was that if I held a button down on my remote I would only receive one MQTT message at the time I released the remote and not a continuous stream of messages for as long as the button was held, RFLink does give multiple messages for a held button. This made it impossible to detect a ‘long press’ which is needed to put the lights into white mode and I’d also like to see the lights change colour as I swipe my finger around the remote’s colour wheel. I also found that the NodeMCU running the MQTT client connected to my RFLink gateway would crash frequently.

This is the NodeMCU firmware I have tried:

How do you use MQTT to receive MiLight instructions without RFLink? The way I did it was to connect a MQTT client onto the serial output of my RFLink gateway. I’m not sure how you do this without the RFLink gateway.

Could you explain what <<: *MILIGHT_PARAMS does please.

Ah, I’m not receiving commands from remotes, actually. It’s in one direction the other way – I send commands via MQTT and the hub translates those to RF packets that the bulbs respond to.

It’s a YAML anchor. They do a good job covering them here: https://learnxinyminutes.com/docs/yaml/. Basically just a way to avoid repeating the same thing over and over.

I think there should be something I can do with the templating for this. However there are some features for rflink I want to work on first (https://github.com/home-assistant/home-assistant/issues?utf8=✓&q=is%3Aissue%20is%3Aopen%20rflink). So maybe I can add initial support and templating later on. I also don’t have milight devices so it is hard for me to test this.

Totally understand, thanks for all your hard work. I can imagine developing for a device you don’t even have any experience with is a nightmare.

I’m currently writing an app with appdaemon that grabs Mi-Light messages from the rflink packets in the HASS log that should do what I want. Slow going as I have to learn appdaemon and python as I go! Hopefully be able to show you what I’m talking about tomorrow. It is my first python program so I can’t wait for you to tell me how terrible it is!