No value updates from Homematic IP

Configuration
1

I am using Home Assistant for a few month now, and so far everything has been working very good. I bought a couple of Homematic IP components recently, and tried to integrate them into Home Assistant (using Raspberrymatic as my CCU3). All components can be integrated and I can successfully change their state (e.g. closing and opening the shutters), so the setup seems to be correct.

But HA never updates the status or values of any of the devices. HA retrieves the state of the devices only once during startup, but the values are never updated again until I restart HA again. I read HA loses the connection if the CCU is rebooted, but even if the CCU is never restarted, HA does not receive any updates.

Can anyone help me? Here is my configuration:

homematic:
  interfaces:
    hm_ip:
      host: 192.168.0.90
      port: 2010
      username: ...
      password: ...
      resolvenames: false
    hm_rt:
      host: 192.168.0.90
      port: 2001
      username: ...
      password: ...
      path: /groups
      resolvenames: json
    hm_groups:
      host: 192.168.0.90
      port: 9292
      resolvenames: json
      username: ...
      password: ...
      path: /groups
  hosts:
    ccu3:
      host: 192.168.0.90
      username: ...
      password: ...
2

Welcome!
Firstly, in your RaspMatic firewall, can you check if you have the relevant ports open and that your HA controller is included in the IP addresses for restricted access.
Secondly, on my configuration the port numbers have a 3 in front of them. I know I changed them some time ago, but I can’t rememeber why! Anyway, it works.

interfaces:
  bidcos:
    host: 192.168.16.13
    port: 32001
    resolvenames: json
    username: !secret homematic_user
    password: !secret homematic_password
  hmip:
    host: 192.168.16.13
    port: 32010
    resolvenames: json
    username: !secret homematic_user
    password: !secret homematic_password
  groups:
    host: 192.168.16.13
    port: 39292
    resolvenames: json
    username: !secret homematic_user
    password: !secret homematic_password
    path: /groups
hosts:
  ccu3:
    host: 192.168.16.13
    username: !secret homematic_user
    password: !secret homematic_password
3

The path: /groups in the hm_rt block is incorrect. And leave out resolvenames in hm_ip if you don’t set it to json. Also, if you only have HmIP-devices, you don’t need the rm_rt section at all.

All those tips above won’t solve your problem though. What @sota said about the different ports might be a step in the right direction. Those are the ports that are being used if you have certificates (ssl) set up on your CCU. So if that’s the case, that might help.
BUT: the fact the you actually get the entities indicates that this isn’t the problem as well. Home Assistant knows about the devices only because it receives a list of devices from the CCU. So in general the communication from the CCU to Home Assistant appears to be working.

One thing you can try is to enable debug loggin for HomeMatic. If you see messages for the state changes, then the data is received at least. You can turn on debug logging like this in configuration.yaml:

logger:
  logs:
    homeassistant.components.homematic: debug
    pyhomematic: debug
4

Thanks for the feedback! I assumed as well that the connection to my CCU should be working, otherwise HA would not be able to retrieve all the devices and their current state during startup.

I activated the debug logging, but I am afraid I dont receive any state change when one of my devices changes any of its values. The only state change I could find was related to the CCU itself:

[homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=homematic.ccu3, old_state=<state homematic.ccu3=0; ${sysVarAlarmMessages}=0.0, ${sysVarAlarmZone1}=, ${sysVarPresence}=True, ${sysVarServiceMessages}=0.0, Aussentemperatur=-20.0, DutyCycle=4.0, WatchDog-Alarm=, friendly_name=ccu3, icon=mdi:gradient @ 2020-10-31T21:12:49.802960+01:00>, new_state=<state homematic.ccu3=0; ${sysVarAlarmMessages}=0.0, ${sysVarAlarmZone1}=, ${sysVarPresence}=True, ${sysVarServiceMessages}=0.0, Aussentemperatur=-20.0, DutyCycle=5.0, WatchDog-Alarm=, friendly_name=ccu3, icon=mdi:gradient @ 2020-10-31T21:12:49.802960+01:00>>
s [b'\x00=\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x05_bp2p\x04_tcp\x05local\x00\x00\x0c\x00\x01']

pyhomematic seems to be logging only the following statements:

2020-10-31 21:21:19 DEBUG (SyncWorker_3) [pyhomematic._hm] ServerThread.getAllSystemVariables: Getting all System variables via JSON-RPC
2020-10-31 21:21:19 DEBUG (SyncWorker_3) [pyhomematic._hm] RPCFunctions.jsonRpcPost: Method: Session.login
2020-10-31 21:21:19 DEBUG (SyncWorker_3) [pyhomematic._hm] RPCFunctions.jsonRpcPost: API-Endpoint: http://192.168.0.90:80/api/homematic.cgi
2020-10-31 21:21:19 DEBUG (SyncWorker_3) [pyhomematic._hm] RPCFunctions.jsonRpcPost: Method: SysVar.getAll
2020-10-31 21:21:19 DEBUG (SyncWorker_3) [pyhomematic._hm] RPCFunctions.jsonRpcPost: API-Endpoint: http://192.168.0.90:80/api/homematic.cgi
2020-10-31 21:21:19 DEBUG (SyncWorker_3) [pyhomematic._hm] RPCFunctions.jsonRpcPost: Method: Session.logout
2020-10-31 21:21:19 DEBUG (SyncWorker_3) [pyhomematic._hm] RPCFunctions.jsonRpcPost: API-Endpoint: http://192.168.0.90:80/api/homematic.cgi

Is there the possibility to check if the CCU actually published the state changes?

5

You might be able to get some hints on what the problem is by logging into the CCU via ssh and looking at the log-files in /var/log. Maybe you’ll find some error messages there.

6

Thanks a lot, I will have a look at the logs. Is there the possibility to change pyhomatic to pool for the values, instead of listening to state changes?

7

No, it’s specifically designed to use the events. Imagine having a shutter-contact at your door to trigger your selfmade alarm, and the sensor only gets polled every minute. That would result in 59 seconds where you can open and close the door without Home Assistant knowing about it. On top of that, this would also cause more radio traffic to get the latest states constantly. So it really shouldn’t be done that way, and therefore will not be implemented.

8

Ok, I understand your point and it is a valid one. Although I think that there are scenarios in which polling would be sufficient, and could even create less network traffic (since there are homematic devices that send a lot of updates).

The CCU log revealed the issue: a timeout while trying to register for the callback. But its interesting that this only happens for my homematic ip interface, the interface for the groups successfully registered.

I set all ports to open and allowed full access to all interfaces. Triggering state changes works as well, it is really weird that only this one step does not work. Do I need to open specific ports on my HA machine for the connection to work?

Nov 1 21:29:08 de.eq3.cbcs.legacy.bidcos.rpc.internal.InterfaceInitializer ERROR [vert.x-worker-thread-2] IO Exception: Could not add interface: homeassistant-hm_ip
de.eq3.cbcs.legacy.communication.rpc.RpcIOException: java.net.ConnectException: Connection timed out (Connection timed out)
        at de.eq3.cbcs.legacy.communication.rpc.internal.transport.http.HttpTransport.sendRequest(HttpTransport.java:110)
        at de.eq3.cbcs.legacy.communication.rpc.internal.rpc.RpcClient.sendRequest(RpcClient.java:94)
        at de.eq3.cbcs.legacy.communication.rpc.internal.rpc.RpcClient.invoke(RpcClient.java:82)
        at com.sun.proxy.$Proxy21.listDevices(Unknown Source)
        at de.eq3.cbcs.legacy.bidcos.rpc.internal.LegacyBackendClient.listDevices(LegacyBackendClient.java:139)
        at de.eq3.cbcs.legacy.bidcos.rpc.internal.DeviceUtil.synchronizedBackendDevices(DeviceUtil.java:142)
        at de.eq3.cbcs.legacy.bidcos.rpc.internal.InterfaceInitializer.handle(InterfaceInitializer.java:109)
        at de.eq3.cbcs.legacy.bidcos.rpc.internal.InterfaceInitializer.handle(InterfaceInitializer.java:26)
        at io.vertx.core.eventbus.impl.HandlerRegistration.deliver(HandlerRegistration.java:212)
        at io.vertx.core.eventbus.impl.HandlerRegistration.handle(HandlerRegistration.java:191)
        at io.vertx.core.eventbus.impl.EventBusImpl.lambda$deliverToHandler$3(EventBusImpl.java:505)
        at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:337)
        at io.vertx.core.impl.TaskQueue.lambda$new$0(TaskQueue.java:60)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:607)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
        at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
        at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:134)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at de.eq3.cbcs.legacy.communication.rpc.internal.transport.http.HttpTransport.sendRequest(HttpTransport.java:106)
        ... 15 more
9

Polling the stated won’t stop the CCU from also pushing the events. Even if you don’t subscribe to them from Home Assistant, the CCU receives them. So no, that wouldn’t save traffic. :wink:

Regarding your problem: if not configured otherwise, Home Assistant will allocate a random port for the incoming connections. But that’s already working since the other device types operate correctly with Home Assistant. And there’s only one server process spawned by Home Assistant. So that shouldn’t be the problem.
Do you have the latest firmware installed on your CCU? Previously there were some problems with IP-devices where the reconnect service didn’t work. That usually shouldn’t be a problem when the CCU hasn’t been restarted. But maybe there’s some other reason why you’re running into a comparable scenario. So maybe the latest firmware can solve the issue.

10

I had a similar issue.

My setup: Raspberrymatic with HMIP components only. 1 group. Firewall disabled/open.
HA got the current values during startup only.
After reading this thread I had a look in the logs of raspberrymatic which told that channel 0 of the wall switch (HmIP-WRC6) does not exists. So I disabled the entitiy in HA. Additionally I added the groups section to the homematic configuration (the group was set in CCU before but not in the yaml). Further I removed the resolvenames: false as stated.
Since then all values were added regularly. But I do not know which change was the successor :slight_smile: