Read Studer parameters via Xcom-LAN and Rest Sensor

Just wanted to post this here as it took several hours of puzling mostly from various non Home-Assistant sources to get the right config together. This information does not seem to be available in these forums yet, so would like to share it here so others don’t have to spend that time.

There are already examples available of how to read Studer (Xtender, VarioTrack, etc) information using a Rest Sensor config. I.e. Studer Innotec components integration

- platform: rest
  name: "Studer"
  unique_id: "studer"
  method: GET
  scan_interval: 30
  timeout: 60
  resource: "https://api.studer-innotec.com/api/v1/installation/synoptic/<system number>/"
  headers:
    Accept: application/json
    UHASH: #SHA256 encoded username 
    PHASH: #MD5 encoded password
  value_template: "OK"
  json_attributes:
    - energy
    - battery
    - power

However, this will only give a limited list of the most common properties.
In my case, I needed information about the AUX1 and AUX2 relais of the Xtender inverter.

In configuration.yaml:
sensors: !include sensors.yaml

In sensors.yaml:

- platform: rest
  name: "Studer Xtender AUX1"
  unique_id: "studer_xtender_aux1"
  method: GET
  scan_interval: 30
  timeout: 60
  resource: "https://portal.studer-innotec.com/scomwebservice.asmx/ReadUserInfo?email=<email address>&pwd=<password>&installationNumber=<system number>&device=XT1&infoId=3031"
  value_template: "{{ 'unavailable' if (value_json.UserInfoResult.ErrorCode!='1') else 'opened' if (value_json.UserInfoResult.UIntValue=='0') else 'closed' }}"
  json_attributes_path: "$['UserInfoResult']"
  json_attributes:
    - UIntValue
    - FloatValue
    - ErrorCode
    - ErrorMessage
    - ScomFormatNo

And add a similar sensor for AUX2 using infoId=3032

You can test your parameters here: ScomWebService Web Service (studer-innotec.com)
The complete list of available infoId’s can be found in appendix.pdf (studer-innotec.com)

It should be fairly trivial to change my sensor above to read or write a parameter instead of reading a user-info.

1 Like

I am trying to run this code in the Development Tools Template debugger but I keep getting the following error.
UndefinedError: ‘value_json’ is undefined

I am able to run the Synoptic Restful sensor block of code in the debugger without errors. Any recommendations?

I am able to read the UserInfo from XT ID 3032 in the Studer Web remote control
image

I can only assume the returned data in your case is not a valid json or xml.

Main things to check is if your parameters for email address, password and system number are right.
For me it helped a lot to first try it on the Studer SComWebService page: https://portal.studer-innotec.com/scomwebservice.asmx?op=ReadUserInfo

You can also just copy the url from within the resource line of your sensor code and paste it inside a web-browser. Should return a value looking like:

<UserInfoResult xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="https://portal.studer-innotec.com/">
  <UIntValue>0</UIntValue>
  <FloatValue>0</FloatValue>
  <ErrorCode>1</ErrorCode>
  <ErrorMessage>OK</ErrorMessage>
  <ScomFormatNo>0</ScomFormatNo>
</UserInfoResult>
1 Like

I was able to get it to work, but had to do a little restructuring and add “- authentication: basic” at the beginning. Home Assistant complained about a duplicate key. Probably due to the existing Synoptic block of code above it.

rest:
# This key appears above in previous block that reads Studer Synoptic data

# Studer Aux contact status read
  - authentication: basic
    method: GET
    scan_interval: 30
    timeout: 120
    resource: "https://portal.studer-innotec.com/scomwebservice.asmx/ReadUserInfo?email=<email>&pwd=<password>&installationNumber=<system_number>&device=XT1&infoId=3031"
    sensor:
      - name: "Studer Xtender AUX1"
        unique_id: "studer_xtender_aux1"
        value_template: "{{ 'unavailable' if (value_json.UserInfoResult.ErrorCode!='1') else 'opened' if (value_json.UserInfoResult.UIntValue=='0') else 'closed' }}"
        json_attributes_path: "$['UserInfoResult']"
        json_attributes:
          - UIntValue
          - FloatValue
          - ErrorCode
          - ErrorMessage
          - ScomFormatNo

image

Do you have more that one block of code to read Studer status info?

I notice that fairly frequently, the portal does not respond back with the data that I request from it. It seems to be a problem with the portal as I also see the same frequent lack of response in the Android app and the Studer portal web app. Sometimes they will stop working for a while, then a little later, they start working again. They mostly work, but not always. When any one stops working, all of the others stop working as well. I don’t think it is just my internet connection as I have heard from another user here in Portugal that they see the same when using the app, as well as other users reporting the same issue in the Czech Republic and in Slovakia. Have you had this experience also?

Good to hear you got it working.

Yes, indeed the Studer portal does not always respond. Usually not a huge problem as these outages usually are very short, up to 15 minutes. The problem definately is on the Studer side and has nothing to do with your own internet connection; on Studer forums there are several complaints about this lack of reliability.

I might spend a little time in the near future to modify my sensor code so that it will just keep using the last known value for a short time (15 minutes or so) instead of immediately going into ‘unavailable’. If it detects the outage is taking unexpectedly long than it can still go to ‘unavailable’ at that point.
It may be sufficient to just set a longer ‘timeout’ value in my existing sensor code to get such behavior, but will need to experiment a little with it.

Yes, that is the loss of response of the Studer portal that I am seeing, which frequently lasts from 5 to 10 minutes. In which forums have you seen others mentioning similar issues?

I have since modified my code so that it uses the last known value indefinitely and indicates when its doing so. I haven’t tried a longer timeout.

This block of code identifies when the portal is not responding and it is displayed in a conditional card that only appears when the portal isn’t responding:

      - name: "Studer Cloud Status"
        unique_id: "studer.cloud_status"
        state: >-
          {% set x = state_attr('sensor.studer', 'energy')['hasInverter'] %}
          {% if x == True %}
            {{'Studer Cloud Online'}}
          {% else %}
            {{'Studer Cloud OFFLINE'}}
          {% endif %}

The Synoptic data returns many data points that I don’t use, so I just use “hasInverter” one of the simpler, non-numeric status ones, to determine if the portal is responding.

The conditional card looks like this:

type: conditional
conditions:
  - entity: sensor.studer_cloud_status
    state: Studer Cloud OFFLINE
card:
  type: entities
  entities:
    - entity: sensor.studer_cloud_status
      secondary_info: last-changed
  state_color: true

The secondary_info: last-changed line tells me how long ago the portal stopped responding. I wish I could also change the text or card color to make it more noticeable, but I haven’t figured that one out yet.

This block of code tests if the cloud is responding, and if not, just re-uses the previous value.

      - name: "Solar Production"
        unit_of_measurement: "kWh"
        unique_id: "studer.solar_prod"
        device_class: energy
        state_class: total_increasing
        state: >-
          {% set x = state_attr('sensor.studer', 'energy')['solar']|float(0) %}
          {% set its_alive = state_attr('sensor.studer', 'energy')['hasInverter'] %}
          {% if its_alive == True %}
            {{x}}
          {% else %}
            {{ states('sensor.solar_prod') }}
          {% endif %}

I recently asked Studer about the two APIs::

Portal API:
After getting the API to work for writing to flash, I notice that there are two portal APIs for accessing the Studer portal data:

  1. Studer Web API this is the API that I have been relying on up to now
  2. ScomWebService Web Service

I have a yaml script that uses both of the above APIs in my Home Assistant installation. Currently I am only using them for reading data. What is the difference between the the two APIs? Is one preferred over the other due to better support, reliability, or other reasons?

This was Studers: response:
Portal API:
The old one (ScomWebService Web Service) will be soon disconnected (1st december 2023).
This API, still good and working leak of security while the one (Studer Web API) encrypt email and password by using MD5 and SHA256 technology, so in fact new one is more safe.

Since Studer will be disconnecting the older API, I have rewritten my RESTful API code blocks and will post them soon, once I work out some final details on the automation that calls RESTful code blocks.

Thanks for that update from Studer. I have now updated my REST requets to no longer use the ScomWebService but to use the Studer Web API instead.

In my case, the code to retrieve the status of AUX1 on the XTender inverter now looks like (in sensors.yaml or rest.yaml):

- platform: rest
  name: "Studer XTender AUX1"
  unique_id: "studer_xtender_aux1"
  method: GET
  scan_interval: 60
  timeout: 900
  resource: "https://api.studer-innotec.com/api/v1/installation/user-info/<installation_id>?device=XT1&infoId=3031"
  authentication: basic
  headers:
    Accept: application/json
    UHASH: !secret STUDER_UHASH
    PHASH: !secret STUDER_PHASH
  value_template: "{{ 'unavailable' if (value_json.status!='OK') else 'opened' if (value_json.uIntValue==0) else 'closed' }}"
  json_attributes:
    - status
    - uIntValue
    - floatValue

With the secret hash codes in secrets.yaml:

###
### Studer Portal and API credentials
###
# Where "PHASH" is the password hashed in MD5
# Where "UHASH" is the email address hashed in SHA256
# Where "userLevelCode" is the code required by the API for Installer QSP level access
STUDER_UHASH: "1234567890abcdef1234567890abcdef1234567890"
STUDER_PHASH: "abcdef1234567890abcdef1234567890"
STUDER_userLevelCode: "123456"
1 Like

Found what I think will be a much better way to read the Studer data, and its done locally through the LAN rather than going through the Studer cloud. I had made arrangements with the owner of the repositoryhttps://github.com/zocker-160/xcom-protocol and he updated his code to add support for communicating with Moxa when in TCP client mode so that the software could be run while still maintaining the system connection to the Studer portal.

I tested the Zocker-160 Xcom-LAN and found that I was able to read and write as expected. The testing is running Xcom -LAN on my laptop by calling it through a loop that writes the output to a csv file that I import into Calc. I seem to be able to successfully read a group of 27 values ​​every on an average of about 2.0 seconds, but at this rate the portal connection gets overwhelmed and stops receiving values, probably due to request collisions. If I add a 5 second delay to the loop, the Studer connection seems to be stable. In a test that ran for just over 5 hours and read 3,000 sets, the time between readings was usually about 7 seconds. It’s faster than the RESTful integration I was using and seems much more reliable. Currently I am using the program on my PC and changing values at the command prompt, and also running value read tests. So far I have only run the read test for about 5 hours, I will try for a longer period. The next step will be to setup an MQTT broker on my PC and clients on both the PC and HA so HA can access the Xcom-protocol data.

If you want to try the python Xcom-protocol program, you can install it using the instructions on the github page: GitHub - zocker-160/xcom-protocol: Python library implementing Studer-Innotec Xcom protocol used by Xcom-232i and Xcom-LAN Below is the python script that I have been using call the program and read data from the Studer. The script includes almost everything that the portal’s Synoptic page includes. The script is run from a command prompt and take two arguments, writes a time stamp and a list of values to a csv file for importing into a spreadsheet. The first argument is the count of how many times to read the data list, and the second argument is how long of a delay in seconds to wait before starting the next read. I recommend setting the delay to 5 or higher if your system is connected to the Studer portal to avoid too many collisions.

Example of using the script:
python3 measurements10.py 10 5
the loop will read 10 values with a delay of 5 seconds after each read.

import sys
import datetime
import uuid
import time

# Your XcomLANTCP and other imports go here
from xcom_proto import XcomP as param
from xcom_proto import XcomC
from xcom_proto import XcomLANTCP


def main():
# Get the number of measurements and delay from command line arguments
    num_measurements = int(sys.argv[1]) if len(sys.argv) > 1 else 1
    delay = float(sys.argv[2]) if len(sys.argv) > 2 else 1

    # Generate a unique identifier for file name
    unique_id = str(uuid.uuid4())

    with open(f'measurement_results_{unique_id}.csv', 'w') as file:
        file.write(f'Time Stamp, ')
 # ENERGY					
        file.write(f'#Has Solar (true/false), ')                   # 1 place holder
        file.write(f'VS-1 PV Solar Energy (kWh), ')                # 2A
        file.write(f'VS-2 PV Solar Energy (kWh), ')                # 2B
        file.write(f'VS-1 Yesterday Solar Energy (kWh), ')	   # 3A
        file.write(f'VS-2 Yesterday Solar Energy (kWh), ')	   # 3B
        file.write(f'#Has Inverter (true/false), ') 	           # 4 place holder
        file.write(f'AC Energy in Current Day (kWh), ')	           # 5	
        file.write(f'AC Energy in Previous Day (kWh), ')           # 6	
        file.write(f'AC_Energy Out Currrent Day (kWh), ')          # 7	
        file.write(f'AC Energy out Previous Day (kWh), ')          # 8	
        file.write(f'#Has Battery Status Processor (true/false), ') # 9 place holder
        file.write(f'Battery Charge (Ah), ')	                   # 10	
        file.write(f'Yesterday Battery Charge (Ah), ')	           # 11	
        file.write(f'Battery Discharge (Ah), ')                    # 12	
        file.write(f'Yesterday Battery Discharge (Ah), ')          # 13	
        file.write(f'#Has AC Coupling (true/false), ')             # 14	place holder
        file.write(f'#AC Coupling (kWh), ')                        # 15	place holder
        file.write(f'#Yesterday AC Coupling (kWh), ')              # 16	place holder					
 # BATTERY					
        file.write(f'Battery Voltage (V), ')                       # 17	
        file.write(f'Battery Current (A), ')                       # 18
        file.write(f'Battery SOC (%), ')                           # 19
        file.write(f'Battery Temperature (°C), ')                  # 20
        file.write(f'Battery Cycle Phase, ')                       # 21			
 # POWER	
        file.write(f'VS-1 PV Solar Power (kW), ')                  # 22A 
        file.write(f'VS-2 PV Solar Power (kW), ')                  # 22B 
        file.write(f'AC Power In (kW), ')                          # 23
        file.write(f'AC Power Out (kW), ')                         # 24
        file.write(f'Battery Power (W), ')                         # 25
        file.write(f'#Has AC Coupling (true/false), ')             # 26 place holder
        file.write(f'#AC Coupling (kW), ')	                   # 27 place holder  
        file.write(f'AC Current In (A), ')                         # 28
        file.write(f'AC Current Out (A), ')                        # 29
        file.write(f'AC Voltage In (V), ')                         # 30
        file.write(f'AC Voltage Out (V), ')                        # 31
        file.write(f'AC Frequncy In (Hz), ')                       # 32
        file.write(f'AC Frequncy Out (Hz)\n')                    # 33 line feed at end
 
        with XcomLANTCP(port=4001) as xcom:
            measurements = 0
            while measurements < num_measurements:
                has_solar = 'NaN'                                                   # 1 place holder
                solar_prod_vs1 = xcom.getValueByID(15017, XcomC.TYPE_FLOAT, dstAddr=701) # 2A
                solar_prod_vs2 = xcom.getValueByID(15017, XcomC.TYPE_FLOAT, dstAddr=702) # 2B
                yesterday_solar_energy_vs1 = xcom.getValueByID(15026, XcomC.TYPE_FLOAT, dstAddr=701) # 3A
                yesterday_solar_energy_vs2 = xcom.getValueByID(15026, XcomC.TYPE_FLOAT, dstAddr=702) # 3B
                hasInverter = 'NaN'                                                 # 4 place holder               
                grid_prod = xcom.getValue(param.AC_ENERGY_IN_CURR_DAY)              # 5
                ac_energy_in_prev_day = xcom.getValue(param.AC_ENERGY_IN_PREV_DAY)  # 6
                ac_energy_out_curr_day = xcom.getValue(param.AC_ENERGY_OUT_CURR_DAY) # 7
                ac_energy_out_prev_day = xcom.getValue(param.AC_ENERGY_OUT_PREV_DAY) # 8
                has_battery_status_processor = 'NaN'                                # 9 place holder 
                battery_charge = xcom.getValueByID(7007, XcomC.TYPE_FLOAT)          # 10
                yesterday_battery_charge = xcom.getValueByID(7009, XcomC.TYPE_FLOAT) # 11
                battery_discharge = xcom.getValueByID(7008, XcomC.TYPE_FLOAT)       # 12
                yesterday_battery_discharge = xcom.getValueByID(7010, XcomC.TYPE_FLOAT)	# 13	
                has_ac_coupling = 'NaN'                                             # 14 place holder
                ac_coupling = 'NaN' 	                                            # 15 place holder
                yesterday_ac_coupling = 'NaN' 			                    # 16 place holder
                battery_voltage = xcom.getValue(param.BATT_VOLTAGE)                 # 17
                battery_current = xcom.getValue(param.BATT_CURRENT)                 # 18
                soc = xcom.getValue(param.BATT_SOC)                                 # 19
                battery_temp = xcom.getValue(param.BATT_TEMP)                       # 20                
                batt_cycle_phase = xcom.getValueByID(3010, XcomC.TYPE_SHORT_ENUM)   # 21 parameter.py is for Vario Track instead of Xtender
                solar_power_vs1 = xcom.getValueByID(15010, XcomC.TYPE_FLOAT, dstAddr=701) # 22A
                solar_power_vs2 = xcom.getValueByID(15010, XcomC.TYPE_FLOAT, dstAddr=702) # 22B
                gridpower = xcom.getValue(param.AC_POWER_IN)                        # 23
                houseload = xcom.getValue(param.AC_POWER_OUT)                       # 24
                battpower = xcom.getValueByID(7003, XcomC.TYPE_FLOAT)               # 25
                has_ac_coupling = 'NaN'		                                    # 26 place holder
                ac_coupling = 'NaN'			                            # 27 place holder
                ac_in_current = xcom.getValueByID(3012, XcomC.TYPE_FLOAT)           # 28	
                ac_out_current = xcom.getValueByID(3022, XcomC.TYPE_FLOAT)	    # 29	
                ac_voltage_in = xcom.getValue(param.AC_VOLTAGE_IN)	            # 30	
                ac_voltage_out = xcom.getValue(param.AC_VOLTAGE_OUT)                # 31	
                ac_freq_in = xcom.getValue(param.AC_FREQ_IN)	                    # 32	
                ac_freq_out = xcom.getValue(param.AC_FREQ_OUT)	                    # 33	

                file.write(f'{datetime.datetime.now()}, ')
                file.write(f'{has_solar}, ')                                        # 1 place holder
                file.write(f'{solar_prod_vs1}, ')                                   # 2A
                file.write(f'{solar_prod_vs2}, ')                                   # 2B 
                file.write(f'{yesterday_solar_energy_vs1}, ')                       # 3A
                file.write(f'{yesterday_solar_energy_vs2}, ')                       # 3B
                file.write(f'{hasInverter}, ')                                      # 4 place holder 
                file.write(f'{grid_prod}, ')                                        # 5 
                file.write(f'{ac_energy_in_prev_day}, ')                            # 6
                file.write(f'{ac_energy_out_curr_day}, ')                           # 7
                file.write(f'{ac_energy_out_prev_day}, ')                           # 8
                file.write(f'{has_battery_status_processor}, ')                     # 9 place holder   
                file.write(f'{battery_charge}, ')                                   # 10
                file.write(f'{yesterday_battery_charge}, ')                         # 11
                file.write(f'{battery_discharge}, ')                                # 12
                file.write(f'{yesterday_battery_discharge}, ')	                    # 13	
                file.write(f'{has_ac_coupling}, ')                                  # 14 place holder
                file.write(f'{ac_coupling}, ')                                      # 15 place holder
                file.write(f'{yesterday_ac_coupling}, ')	                    # 16 place holder
                file.write(f'{battery_voltage}, ')                                  # 17
                file.write(f'{battery_current}, ')                                  # 18
                file.write(f'{soc}, ')                                              # 19
                file.write(f'{battery_temp}, ')                                     # 20
                file.write(f'{batt_cycle_phase}, ')                                 # 21
                file.write(f'{solar_power_vs1}, ')                                  # 22A
                file.write(f'{solar_power_vs2}, ')                                  # 22B
                file.write(f'{gridpower}, ')                                        # 23
                file.write(f'{houseload}, ')                                        # 24
                file.write(f'{battpower/1000}, ')                                   # 25
                file.write(f'{has_ac_coupling}, ')			            # 26 place holder
                file.write(f'{ac_coupling}, ')			                    # 27 place holder
                file.write(f'{ac_in_current}, ')                                    # 28	
                file.write(f'{ac_out_current}, ')	                            # 29	
                file.write(f'{ac_voltage_in}, ')	                            # 30	
                file.write(f'{ac_voltage_out}, ')                                   # 31	
                file.write(f'{ac_freq_in}, ')	                                    # 32	
                file.write(f'{ac_freq_out}\n')	                                    # 33 line feed at end
#                print(measurements)                              
                time.sleep(delay)
                measurements += 1 
                

if __name__ == "__main__":
    main()
1 Like

Excellent work!
Being able to get Studer values via LAN will greatly improve reliability as the Studer web api continues to have many (though short) outages.

Your suggested method of keeping your script on your pc, write the values to MQTT and read them from there in HA will work.
A simpler method might be to have a HA automation that periodically runs your script to retrieve the XCom values and write into a (json) file. Then you can create your own template sensors that retrieve their value from that file.

For the longer term it would be great to have a ‘proper’ HA integration module for Studer.
Am still working on another Home Assistant project at the moment, but will put this on my list to implement as a possible ‘custom integration’ (which can then become a default integration when stable). Having the XCom python library available clears a big hurdle that was blocking that.
Tricky part will be to figure out how to let people configure which XCom ID’s to retrieve. Not sure if I can solve that puzzle.

The work of creating the program wasn’t mine. It was created by Zocker-160 on github. There was another by mustafaabughazy, but I preferred the one by Zocker as it seemed simpler and had the ability to write to Studer RAM. Originally his program only included serial and UDP communications, but he added TCP client at my request so that it would work without having to disconnect the PV system from the Studer portal.

Running the python Xcom program on my PC is an intermediate step. I don’t have experience with MQTT yet, so I thought it would be easiest to get started there. I was looking to use eventually run the Xcom in a Docker container in the the same Pi that is running my HA and pass the information using MQTT as I don’t know of a way to have an HA automation run a python program.

The communication works most efficiently when given a list of User Infos or parameters to read rather than trying to request them individually. Would it be possible for you to provide a list in the integration for the end user to populate with those Xcom IDs and device addresses that they are interested in reading from. Writes would probably be done individually as those probably don’t need to be frequent and wouldn’t need to be many different IDs.

I did some further work on the Studer script after the owner of the Xcom library on Github shared a function that he made for the python script to write to HA. The link to the script and a write up is here. Access Studer-Innotec Xtender PV system using local communications - #8 by JeffersM