Custom modbus integration (AVE)

Hi,

I am new to Home Assistant and I am seeking for advice.

At my home I have installed a smart home system produced by AVE (an italian company) which handles lights (switches), covers (rollershutters) and thermostat trough an hub.

I want to integrate this system with HA, but there is no AVE integration yet; luckily it is possible to interface with ave hub with the modbus protocol. I have already positively tested some proof of concept code written in python (with pymodbus).

Sadly the official modbus integration shipped with HA doesn’t quite fit the schema provided by ave; for example the schema for the covers provided by the modbus integration provided only 4 states (state_open, state_closed, state_opening, state_closing) while the ave implementation has 5 different states (is_open, is_closed, is_opening, is_closing, is_halfway) and 3 different commands to control the cover (open, close, stop) on the same resister.

The modbus.switch integration almost works, but for a stupid bug on ave’s side, i need to issue any command twice, with a short delay between, otherwise the state of the switch is not correctly updated (the first command effectively turn on/off the switch, but the register used to represent the state does not change accordingly; the second command just force an update on the register and then physical state and digital state are aligned, all is fine in the world)

My question is, since I am new to this stuff, what is the best strategy to tackle this custom integration issue? Should I write my own custom integration, or there are other ways to achieve such result within HA?

I was thinking about copying and adapt the modbus integration repository available on github to fit my needs, and add it as a custom integration on my HA server.

By the way I don’t understand why there is just a write_register/coil service available on the modbus integration and no read_register/coil service; with just those two services you can do everything! I think I could write my own custom switch within automation or script if read_register was available.

What’s your thought, community?

Did you succeeded with this?

No, I didn’t do anything about the modbus integration.

In the end I preferred to chose the mqtt integration: it is quite flexible and can integrate stuff without writing custom code. By the way I discovered the AVE is mqtt enabled later, by chance, since it is an undocumented feature.

Hello Kompre,
I installed the same AVE system as you and I am looking for an HA integration.
Are you using MQTT at the end? is it working well or do you trouble with it?

MQTT is the way to go: you can easily handle lights, blinds, and thermostat.

Just need to configure the mosquitto broker so that it point to the IP of the ave server, and port is usually 1883. I don’t think you even need username and password, but in case, they are stored on a text file inside the ave server itself, you just need to connect via ssh.

By the way, is the clock on your ave server loosing time each day? Mine was accumulating a couple of seconds of retard each day so that in a couple o week the clock was utterly unsynced. It was bad…

thank you for the MQTT suggestion, I will give it a try and let you know.Unfortunately I have no user and password for the ssh but I will look around to find them.
My clock is OK.I installed the last version of the server just 1 year ago, maybe you have an older one?it is very strange to lose ticks on a PC clock…

probably the default credential are enough to ssh into… give it a try!

I don’t think you strictly need those tough, once you set up the mqtt broker, you can map your plant simply by listening for event and try to turn on and off your various devices:

  • lights are easy, simple on and off
  • thermostats have some more settings, but they are quite easy to configure: they will publish the current temp constantly, and you can set on/off and target temp with the mqtt configuration.
  • blinds are tricky: the mqtt commands to close/open are the same of the current state of the blind, BUT the blind won’t update its state until it completes the cycle, i.e. if you issue a command to close the blind, the blind, if asked, will report an open state until it is fully closed (it’s time based, you usually hear a click few seconds after the blind is physically closed); if you stop the blind halfway, it will still report as open (or close, depending on the start position).

the blinds gave me some trouble to implement, because that behavior caused some “ghost” command, with the blind “stuttering”, but I found a nice trick and workaround and they work flawlessly. If you are interested, I can publish an example mqtt configuration for each device.

Mind you, this features are undocumented, they works for me, but your application may vary.

1 Like

Thank you very much for your suggestions Kompre!! it is much more of what I’ve found around for month!
I never used mqtt but it shoul dnot be a big challenge…I will let you know if I can capture the Domina events!
Thank you !

So, I’ll paste my own mqtt configurations for lights, thermostat and blinds. I am assuming you already know your way with yaml and the config.yaml file. I’ve enclosed in < > what the fields that should be filled with your own ave installation:

  • <MAC ADDRESS>: I think it’s that mac address of the ave server; you can find it easily listening for any event on the mqtt integration and trying to switch on and off something; it’s the same for every device;
  • <id>: it’s the id that ave assign to each device in your installation; as above, you can sniff the id by switching on and off the relevant device in your installation;
  • <unique_id>: the HASS unique id for the device; must be lowercase and without any space
  • <friendly name>: the name that will be shown on HASS interface; it can be aby string you like

NOTE! the mqtt integration will let you to set the retain policy: NEVER SET RETAIN TO TRUE!.

if you set retain to true, the mqtt server will remember the last imparted instruction, and republish it whenever the server get disconnected/reconnected (which happes quite often), therefore you can have some “ghost” commands fired randomly in inconvenient time (like in the middle of the night, scaring me a lot!). Just let retain to false and never be bothered back.

Lights (switches):

lights are pretty straightforward, just a simple on/off command. In the eye of HASS, this are really switches more than lights.

switch:
- platform: mqtt
    unique_id: <unique_id>
    name: "<friendly name>"
    state_topic: "/<MAC ADDRESS>/devices/lights/<id>/state"
    command_topic: "/<MAC ADDRESS>/devices/lights/<id>/state"
    payload_on: "1"
    payload_off: "0"
    optimistic: false
    qos: 0
    icon: hass:lightbulb # comment out if you want simple switch icon instead of the lightbulb

Thermostat (climate)

thermostat are quite basic: they will constantly publish current temperature, you can read and set the target temp, and you can turn it on and off.

You can’t change the schedule like in the ave app, but with these tool you can set up one of your own with HASS if you like.

climate:
  - platform: mqtt
    unique_id: <unique_id>
    name: "<friendly name>"
    modes:
      - 'off'
      - 'heat'
    current_temperature_topic: "/<MAC ADDRESS>/devices/thermostats/<id>/current_temp"
    temperature_command_topic: "/<MAC ADDRESS>/devices/thermostats/<id>/target_temp"
    temperature_state_topic: "/<MAC ADDRESS>/devices/thermostats/<id>/target_temp"
    power_command_topic: "/<MAC ADDRESS>/devices/thermostats/<id>/state"
    payload_on: 1
    payload_off: 0
    qos: 0

Scene

you can even turn on the scene programmed in the ave server. (I am using this less and less though)

scene:
  - platform: mqtt
    unique_id: <unique_id>
    name: "<friendly name>"
    command_topic: /<MAC ADDRESS>/devices/scenes/<id>/request
    payload_on: RUN
    qos: 0
    retain: false

Covers

And now I present you my pride and joy!

Covers are tricky to do with mqtt because the topic to set the state of the blind is the same one where to read the current state. So if you want to query the state of the blind, you are really issuing a command to the device. Also the ave server won’t update the state of the blind until it has complete the cycle (if you send open command, the ave server will report the cover is closed until it is fully opened and you hear the physical switch goin off; same in the other direction).

Furthermore, the ave server only support two states: 0 (closed) or 1 (open), but in reality the cover will be in intermediate state (opening and closing) and also you can decide to stop it halfway.

Combine all of this together and you get false commands and “stuttering” of the blind. I strongly suspect that’s the reason you can’t control a cover with google home or alexa.

So, how you can control a device with only two possible states with the HASS cover integration which support 4 different states (5 if you count the stop halfway command)?

Here comes the trick: ave server will only care for the first character of the message payload published on the state topic, so you can have different messages for the different states as long you reseverved the first character to either be a 1 or 0 so that ave server can handle it; HASS will instead read the rest of the message and set the cover state accordingly. I’ve used three payload command (0-OPEN, 0-STOPPED, 1-CLOSE), and five different states:

  • state_open: 0 (will be pulished by the ave server when cover is fully open)
  • state_opening: 0-OPEN
  • state_stopped: 0-STOP
  • state_closing: 1-CLOSE
  • state_closed: 1 (will be pulished by the ave server when cover is fully closed)
cover:
  - platform: mqtt
    unique_id: <unique_id> 
    name: "<friendly name>"
    state_topic: "/<MAC ADDRESS>/devices/blinds/<id>/state"
    command_topic: "/<MAC ADDRESS>/devices/blinds/<id>/state"
    position_topic: "/<MAC ADDRESS>/devices/blinds/<id>/state"
    payload_open: "1-OPEN"
    payload_close: "0-CLOSE"
    payload_stop: "0-STOP"
    state_open: "1"
    state_closed: "0"
    qos: 0
    device_class: shutter
    state_opening: "1-OPEN"
    state_closing: "0-CLOSE"
    state_stopped: "0-STOP"
    optimistic: true
    position_template: >- # with this you can differentiate the state when the cover is stopped halfway
      {% if (value == '0') %} # the cover is fully closed (0%)
      0
      {% elif (value == '1') %} # the cover is fully open (100%)
      100
      {% else %} # any other position will be report at 50% (halfway)
      50
      {% endif %}

Conclusion

That’s all I was able to to do with the mqtt integration, and that’s more than I could ever do with the ave app or with alexa/google. Have a look if this can be helpful with your installation.

I also have an economizer device in the ave system, but for that you can’t use mqtt; you need the REST integration.

Superb!!
you’ve done a lot of work around it! that’s really a great integration.
I keep your post in mind for my tests, 'cause I am still trying to send a mqtt message to the server :frowning:
I’ll keep you posted.
Thank you!

A little of progress here.
mqtt integration is enabled and configured with IP address of the server in HA
mqtt is enabled on Ave server and service is listening on default port
Mac is already in my hands.
I added your lights configuration to my configuration.yaml for testing purpose, filled some testing <unit_id> and tried to turn on the light .
unfortunately nothing happens.
here my config:

switch:
  - platform: mqtt
    unique_id: 123
    name: "luce"
    state_topic: "/00:A3:33:0A:79:32/devices/lights/<id>/state"
    command_topic: "/00:XX:3A:0D:65:86/devices/lights/<id>/state"
    payload_on: "1"
    payload_off: "0"
    optimistic: false
    qos: 0
    icon: hass:lightbulb

I also tried to send topic like:
mosquitto_pub -h <ipaddress>-t home-assistant/switch/1/on -m "switch is ON"
but no lucky.

sure I am missing something…but what? can you put me in the right direction?
thanks

Ok, I am seeing some easy to fix mistake

The mqtt server is on the AVE server: in the broker option of the mqtt integration you should set the IP of the AVE server, not HA. In the Configuration>Integration section, you click on the configure button of the MQTT integration

On the following screen you should see three section:

  1. configuration settings
  2. publish a packet:it’s equivalent to the mosquitto_pub command line tool and it is used just for testing
  3. listen to a topic: this is equivalente to the mosquitto_sub command line tool and also it used just for testing

Go on the first one and configure the mqtt: just insert the IP address of the AVE server and port 1883 (default) and you should be fine (I don’t think you need username and password, at least I don’t):

image

If the configuration is successful you should be able to listen for any message published on the mqtt server. Test it by typing # in the Listen to a topic section (# is a wildcard for mqtt that means anything). If you prefer, you can also use the command line tool:

mosquitto_sub -h <IP AVE SERVER> -t '#' -v

Once you start listening to events, they should appear as soon as they are happening: try to switch physically ON and OFF lights or whatever device you have and look for the message, fiddle a bit to understand how the system works

image

Be aware that with the # wildcard you’re listening to all events published on the mqtt server, and sometimes could be messy: if you want to listen for just a family of device, or even a single device just change the topic from # to something more specific like /<AVE mac address>/devices/lights/# or /<AVE mac address>/devices/lights/<ID>/#

If you can reproduce all of these, you’re almost good to go.

Also be careful, here you did a mistake:

the mac address is always the same and should be the AVE server mac address

If you have correctly set up the mqtt integration to listen to the AVE Server, you can try to publish a message: just copy the topic for a device and publish a valid message:

mosquitto_pub -h <IP AVE SERVER> -t '/<AVE MAC ADDRESS>/devices/lights/16/state' -m '1'

If you’re successful, now it came the tedious part: map your home.

You can either decide to simply manually switch device on and off while listening to the mqtt server and take note of the ID of each one, or, if you know what are you doing, retrieve the configuration list from the ave server itself via ssh (it’s a xml file called configuration.xml somewhere in the ave server; have a look in /var/DPData/, but yours could be different, I’m not sure)

Let me know how it goes

thank you for your help.
I did not understand I had to subscribe all the topics to see command in the Hass UI !
I fixed some mistakes too :slight_smile:

Switches are ok and work very well even if I preferred to use the lights.
Also thermostats work well but now the tricky part…Covers!

I am able to issue the command to move the blind up (1) and down (0) but that’s all I can do.If I send 2 times the same command I can also stop the blind in a kind of intermediate state.
As you wrote, the Ave server seems not update the intermediate status via mqtt but anyway, that status is present in the Ave app as described by the icons that the system provide.
opened Screenshot from 2021-10-01 17-24-56
closing Screenshot from 2021-10-01 17-25-10
closed Screenshot from 2021-10-01 17-25-20
and the intermediate status: Screenshot from 2021-10-01 17-24-32

Is it just a not published topic on mqtt or there is something more behind? what’s your thought?
another thing…I am not sure I understood this part of the yaml:

    position_template: >- # with this you can differentiate the state when the cover is stopped halfway
      {% if (value == '0') %} # the cover is fully closed (0%)
      0
      {% elif (value == '1') %} # the cover is fully open (100%)
      100
      {% else %} # any other position will be report at 50% (halfway)
      50
      {% endif %}

if the Ave server manage only 0 and 1 how the else condition can be verified?
Sorry if I made a stupid question… :frowning:

thank you for everything!

In HA a switch is a entity with just two state (OFF/ON or 0/1), whenever a light can have more state like RGB value, dimmer value, temperature, etc; basically a light is the ectension of a switch. In AVE case most of what we call lights are really simply switches. I’ve used the icon: hass:lightbulb in my configuration so that the switch will shows with lightbulb icon, as I prefer (and also I can differentiate with proper switches I have in my home, not connected to a bulb)

Yeah, Covers will not publish intermediate state on the relevant topic, and indeed they were the most troublesome to integrate. What my integration is doing is mostly an hack to get five states from just two outputs… But to be clear, you as the user operating the cover from HA will only publish on the topic 3 possible commands:

and the AVE server will publish 1 or 0 when you physically use the switch on the wall, or when the cover finishes is opening/closing cycle. Thus we have 5 states: 3 new states from within HA and 2 original states from AVE:

#
# 3 states that generates only from within HA:

    state_opening: "1-OPEN" 
    state_closing: "0-CLOSE"
    state_stopped: "0-STOP"

# 2 states orginal to ave server:
    state_open: "1"
    state_closed: "0"

All the above states are published on the same topic, which is also the topic where the mqtt integration will read the position value. Hence this bit of code:

The else condition is verified for the state_opening, state_closing and state_stopped, which are different from 0 and 1.

Probably you can encounter some issue with the reported state of a cover in HA if you manually operate a cover, but this false state will eventually fix itself whenever the cover is fully opened/closed.

And also, I barely touch those switches anymore :smile:

I’m glad you were able to use this integration, it will boost your satisfaction with the ave smart home system by a lot, trust me. I almost never open anymore the AVE app, which, quite frankly, sucks.

By the way, do you also have this thing?

This device will not publish on the mqtt server. If you want to be able to read the actual energy consumption, you need to use the rest integration. Well I guess I paste my code:

sensor:
  - platform: rest
    name: economizzatore
    resource: http://<AVE SERVER IP>/bridge.php?command=GEC
    value_template: "{{ value_json['root']['records']['record']['dato1'] }}"
    unit_of_measurement: kW
    # scan_interval: 10
    # force_update: yes
    device_class: power

Ooook very good explanation, thank you!
I trained myself a bit more on switches and cover, I had some gaps :slight_smile:
I do not have electricity consumption reader but I am really interested in your code…
If you do not mind…where/how did you find the REST commands to query the server?

Grazie mille di tutto :wink:

you can find the code in the var/www of your AVE server. You should find a .js file called remote with all the commands listed. Basically it is what the web interface is doing and with the rest command you’re emulating that.

Then it is a matter of poking in the code and learning what does what.

It is possible to set the integration with REST, but I find it’s a little bit too slow compared to MQTT (MQTT could be classified as push local, instead a REST integration would be pull local).

prego!

Hello there… :slight_smile:
Is still someone trying to implement integration between AVE and Home-assistant ? i try the MQTT way and work well but is an unsupported way and AVE refuse to give any assistance on it.
And on other side i need the MQTT dispatcher for another integration.
So i try the Modbus way… (AVE in this case give some support :slight_smile: ) but im stuck…

In anyone is still interested we can start a discussion…

you can have a bridge configuration, so that your main mqtt server is hosted on HA, while sending and receiving only the relevant mqtt message to ave server. This way you can have other mqtt integration that do not interfere with ave stuff.

Modbus is tricky because the ave schema is not fully compatible with HA, it requires some work. I’ve only migrated the covers control to modbus because so I can know the intermediate states like opening and closing.

What device do you have? switches and covers?

I have light switchs (442ABT4) , menaged plugs and heater pump controls (53ABRTM-PV).
(i also have covers but they are not connected , reserved for future use).

The MQTT bridge is a good idea… but i fear that AVE could close the MQTT access in the future.

My first objective is to let work the light switch to allow some device to turn on and off depending on how much the solar PV is producing… :stuck_out_tongue:

The only way AVE could close the mqtt is for you to install a new firmware, and I really doubt you’ll get in touch with them again… I wouldn’t worry much, you’re in control of your installation, and, if you are really worried about them pushing an update without your consent, just disconnect ave server from the internet. Once you have HA managing all, you’ll never open ave app again.

Also make a backup of your configuration if you can.