Thank you for your explanation. I thought, it would be much easier to implement this feature. 
Too bad, that my coding knowledge is nearly zero. Well, i have to stay at z2m then i think. 
@le_top Thanks again for continuing to develop zha-toolkit! Wondering if can suggest that you update your original post in this thread for the current feature so that it summarize is correct when it is linked:
HA2023.7 with zigpy 0.56.0 has breaking changes, you need to update to zha-toolkit v0.9.4Can “responding services” feature from Home Assistant Core 2023.7 make ZHA Toolkit more useful?
https://www.home-assistant.io/blog/2023/07/05/release-20237/#services-can-now-respond
Sounds also like this feature could make it easier to use ZHA services directly in an automation?
Hi @Hedda
zha-toolkit did not wait for such a feature for providing responses which can be fouind in the event data. There are some examples that demonstrate how to use that.
However, it would certainly beneficial to provide the data directly to the automation rather than having to retrieve it from the event (which is kind of complex to capture in an automation).
However, I did not find technical documentation on how to generate those responses in a component’s python by following the links in the release notes.
Home Assistant Developers Blog only mentions “service return values” which might be the same thing?
https://developers.home-assistant.io/blog/2023/06/14/service-calls/
https://github.com/home-assistant/architecture/discussions/777#discussioncomment-6127898
I did find this example on “Response Data” for services in the Home Assistant documentation:
https://www.home-assistant.io/docs/scripts/service-calls#use-templates-to-handle-response-data
and this about “service response data” in the developer’s documentation:
https://developers.home-assistant.io/docs/dev_101_services/#response-data
Release notes also mention that integration developers first need to enable it per service. So you can not get responses from just any service, instead, it has been enabled in the integration for each specific service that you want to be able to give a response.
https://developers.home-assistant.io/docs/dev_101_services/#supporting-response-data
As I understand, the release notes implies that other than for scripts it has so far only been enabled for calendar.list_events and conversation.process services in the Calander integration as an example so guess that integration developers would need to use that as a reference?
Perhaps @allenporter has more info or links on implementing responses for services in integrations?
You found everything… The last link is the developer docs for implementing this.
@Hedda Thanks for pulling this together.
@allenporter I agree that Integration Services | Home Assistant Developer Docs holds the relevant information.
The documentation suggests:
hass.services.async_register(
DOMAIN,
SEARCH_ITEMS_SERVICE_NAME,
search_items,
schema=SEARCH_ITEMS_SCHEMA,
SupportsResponse.ONLY,
)
but then I get SyntaxError: positional argument follows keyword argument - I will find the correct way to do it (named argumetn or positional argument).
Submitted docs: fix Supporting Response Data example by pdecat · Pull Request #1853 · home-assistant/developers.home-assistant · GitHub to fix the documentation example.
zha-toolkit is now updated to provide response data and I updated one of the scripts to use it:
The response data is also provided by when calling the service interactively which is handier than having to check the even_data.
3 Likes
Hi
I’m trying to use Toolkit, which has been installed/activated correctly. I also activated the custom component under “log” in my configuration. I understand that the feedback from running the services will be listed in the Home Assistant core log file - correct? The problem is I see nothing in the log file.
Is there somebody who can point me in the direction of a solution?!
One way to verify that zha-toolkit is correctly set up is to verify that they appear in “Developrer Tools > Services” (example in French):
Using that same location to run services, you can dynamically change the log level (no need to add them to you static configuration - the configuration is kept until reboot or another change):
service: logger.set_level
data:
custom_components.zha_toolkit: debug
zigpy.zcl: debug
It is true that the traces should occur in the log (/config/home-assistant.log).
With Home Assistant >=2023.7 and zha-toolkit>=v0.9.7, you can also see the response when using th e “Developer Tools>Services” without the need to activate logging or listen to events:
And when using a zha-toolkit service, you can get the result in a variable by adding “response_variable: …” to the service arguments.
Do you see the zha toolkit services? Do you have zha insdtalled as well (zha-toolkit ‘extends’ zha)? zha has services as well, so you can check that you have access to those:
@le_top have you looked at zha-toolkit using “Manufacturer OTA Firmware Update Support” (also referred to as “Generic OTA providers”) that was added with zigpy 0.56.3 release in HA 2023.8.1?
https://github.com/zigpy/zigpy/pull/1165
This allows for new OTA providers to be added to zigpy without extra code, which in turn also allows for users and manufacturers to distribute test feed URLs to customers without deploying them globally.
https://github.com/zigpy/zigpy/wiki/OTA-Information-for-Manufacturers
I believe that this could for example enable users to easier add to for example grab Zigbee OTA firmware files from Koenkk’s Zigbee OTA repo → https://github.com/Koenkk/zigbee-OTA/blob/ef0e28e5bdd0d5c6dca0bed4e5230b86bdea3e2c/index.json (as requested here → https://github.com/zigpy/zigpy/issues/535)
PS: puddly also added a new feature to the “zigpy-cli” tool that manufacturers can use to produce/generate a skeleton index from a given set of OTA files, and that way makes sure manufacturers OTA files are correctly-structured so that they will be compatible with this new JSON format for zigpy OTA providers out-of-the-box. Check out the README.md for the “zigpy-cli” (zigpy CLI tool) → https://github.com/zigpy/zigpy-cli/blob/dev/README.md#generate-ota-index-files
Thank you for your detailed response.
It looks like the Toolkit is installed correctly. I will give it a go again, using your inputs.
Thanks. - Thomas
zha_toolkit.scan_device is not reporting attributes for manufacturer cluster. In the scan file the cluster is there but for attribute I get only “attributes”: {},
Attributes are there as I can read there value in ZHA but none are listed in the scan. Is there something I need to do to improve scanning
IHMO the device is not reporting the attributes for the manufacturer clusters.
I have the result of a zha_toolkit.scan_device for Danfoss/eTRV0100 that also has the the details for the manufacturer attributes in the alongside the normal attributes.
Example:
"0xfffd": {
"attribute_id": "0xfffd",
"attribute_name": "cluster_revision",
"value_type": [
"0x21",
"uint16_t",
"Analog"
],
"access": "REPORT|READ",
"access_acl": 5,
"attribute_value": 1
},
"0x4000": {
"attribute_id": "0x4000",
"attribute_name": "sw_error_code",
"value_type": [
"0x19",
"bitmap16",
"Discrete"
],
"access": "REPORT|WRITE|READ",
"access_acl": 7,
"manf_id": 4678,
"attribute_value": 0
},
I know that others don’t report them and I also know that at least one major vendor of zigbee components does not natively propose features in at least one of its zstack frameworks allowing to easily add manufacturer attributes so that they are properly reported: it’s up to the developer to re-implement all that… So given the amount of effort that represents, manufacturer attributes may not reported.
Ok so the only way to find out the attributes of the manufacturer cluster is to sniff the zigbee data ?
There is another method, slower requiring less effort and maybe faster if you consider the time you have to spend to decipher the communications:
To the clusters that you know exist and those that you think that exist read all attributes with the manufacturer id set. Readable attributes should reveal their type, write-only attributes may reveal they exist (not sure what error is reported in that case).
You can start from the script that reads several attributes in the basic cluster available as an example: https://github.com/mdeweerd/zha-toolkit/blob/c10f0995eb763accbedd9416af34fdf519f37e05/examples/script_read_basic_cluster.yaml .
You can adapt it to read another cluster (cluster<>0), more attributes (0 to 0xFFFF for the ‘full’ range - not sure that the last values are valid attirbutes), and add the manufacturer_id as a parameter to the reads.
You can stop the script when you think you found them all - bit some manufacturers will start their attributes with a high id such as 0xFC00 .
If you can read an attribute every second, then you can read all of them in about 18 hours.
Some devices do not respond when the attribute does not exist, so then it takes longer, and with sleepy devices it takes even longer.
However, patience will take you there and you can also accellerate if you can make smart guesses for the ranges.
The results are written to a CSV file - so you can discover what could be read and what not.
I’ve test ed as below but attribute 3 or any other I’ve tested are not there
warnings:
- “Result: {3: 0} - Attribute 3 not in read ({}, {3: 0})”
success: false
service: zha_toolkit.attr_read
data:
ieee: 50:0b:91:40:00:02:26:6d
manf: 17672
endpoint: 1
cluster: 65281
attribute: 3
event_success: my_read_success_trigger_event
event_fail: my_read_fail_trigger_event
event_done: my_read_done_trigger_event
csvout: ../www/test.csv
do I miss something
When the read returns something, you would see this in the response:
result_read:
- "3": 1
- {}
while you have:
result_read:
- {}
- "3": 0
Which means that there was no value for the attribute, however there also was no error code (0 means everything is ok). “No value” might mean “empty value”, but I’ld prefer to check that (see below). As you can sniff as far as I understand it, this is an interesting case.
When I read an attribute that does not exist, I get this:
result_read:
- {}
- "30": 134
warnings:
- >-
Result: {30: <Status.UNSUPPORTED_ATTRIBUTE: 134>} - Attribute 30 not in read
({}, {30: <Status.UNSUPPORTED_ATTRIBUTE: 134>})
Which shows that 134 is reported as an unsupported attribute.
Do you have “attr_type” in your response? If you do, share the type. Maybe it’s a field that can be empty (zero length) and then I need to improve handling of this kind of fields.
Otherwise I would conclude that the device is not responding correctly and that the field is a write only field.
On my end, I can not find a device with an attribute that is WRITE only, so I can’t compare.