🛡 Watchman - keeps track of missing entities and services in your config files

OzGav · February 21, 2023, 10:58pm

Is there anything in the config ignored options? How many entities are shown if you do what Tom suggests four posts above?

PlayFaster · February 22, 2023, 2:07am

Hi There

This works for me, after some modification, to display missing services. Breaking out all of the lines is not required, it just helped me for clarity.

Markdown to display WatchMan report missing services …

  - type: markdown
    content: >-
      <h2>
      <ha-icon icon='mdi:shield-half-full'>
      </ha-icon>
      Services report
      </h2>
      <h3> Missing Services:
      {{ states.sensor.watchman_missing_services.state }}
      </h3>
            {%- for item in state_attr("sensor.watchman_missing_services","entities") %}
      <hr>
      <table>
      <tr>
      <td>
      {{ item.id }}
 
      <a title="{{item.occurrences}}">
      {{item.occurrences.split('/')[-1] }}
      </a>
      </td>
      </tr>
      </table>
      {%- endfor %}
    card_mod:
      style:
        ha-markdown:
          $: |
            ha-markdown-element:first-of-type hr{
              border-color: #303030;
            }

Haze · February 22, 2023, 10:31pm

This worked or me. Thank you!

TRusselo · March 2, 2023, 5:16pm

I have added a few things to make my Watchman Card more helpful to me:
Tracking the number of ZWave Devices alive vs. connected.
Tracking the number of Zigbee2MQTT devices.

sensor.yaml

  - platform: template
    sensors:
       count_zwave_devices:
        friendly_name: 'ZWave'
        icon_template: mdi:z-wave
        value_template: >
          {% set total = states|selectattr('object_id', 'search', 'node_status')|list|count %}
          {% set dead = states|selectattr('object_id', 'search', 'node_status')|selectattr('state', 'in', 'dead, unavailable, unknown') | list | count %}
          {{ total - dead }} Alive/{{ total }} Total

      count_zigbee_devices:
        friendly_name: 'Zigbee'
        icon_template: mdi:z-wave
        value_template: >
          {% set entities = integration_entities('mqtt') %}
          {% set ns = namespace(unique_ids = []) %}
          {% for entity in entities %}
            {% if device_id(entity) not in ns.unique_ids %}
              {% set ns.unique_ids = ns.unique_ids + [device_id(entity)] %}
            {% endif%}
          {% endfor %}
          {{ ns.unique_ids | count -2 }}  ## -2 was added for my config to be accurate, yours will differ. i have 2 mqtt devices that are not zigbee

Watchman Report Card:

<h2> <ha-icon icon='mdi:shield-half-full'></ha-icon> Watchman report</h2> <h3>Missing Entities: {{ states.sensor.watchman_missing_entities.state }} </h3> {%- for item in state_attr("sensor.watchman_missing_entities", "entities") %} <hr> <table><tr> <td> <ha-icon icon='mdi: {%- if item.state=="missing"-%}cloud-alert' {%- elif item.state=="unavail" -%}cloud-off-outline' {%- else-%}cloud-question' {%- endif -%} ></ha-icon> {{ item.id }} [{{item.state}}] <a title="{{item.occurrences}}"> {{item.occurrences.split('/')[-1].split(':')[0]}}</a> </td></tr></table> {%- endfor %}
<h2> <ha-icon icon='mdi:z-wave'></ha-icon> ZWave report</h2> <h3>Devices: {{ states.sensor.count_zwave_devices.state }}
<h2> <ha-icon icon='mdi:zigbee'></ha-icon> Zigbee report</h2> <h3>Devices: {{ states.sensor.count_zigbee_devices.state }}

Ronnie-J · March 9, 2023, 1:41pm

Does anyone know why I keep getting an error trying to get Watchman to send me a telegram notification?

Trying via the developer panel caling the report service;

service: watchman.report
data:
  create_file: true
  send_notification: true
  service: notify.telegram
  data:
    parse_mode: html
  parse_config: true
  chunk_size: true

Logger: homeassistant.helpers.script.websocket_api_script
Source: custom_components/watchman/__init__.py:422
Integration: Watchman ([documentation](https://github.com/dummylabs/thewatchman), [issues](https://github.com/dummylabs/thewatchman/issues))
First occurred: 2:37:59 PM (2 occurrences)
Last logged: 2:38:31 PM

websocket_api script: Error executing script. Unexpected error for call_service at pos 1: the JSON object must be str, bytes or bytearray, not dict

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 451, in _async_step await getattr(self, handler)() File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 684, in _async_call_service_step await service_task File "/usr/src/homeassistant/homeassistant/core.py", line 1808, in async_call task.result() File "/usr/src/homeassistant/homeassistant/core.py", line 1845, in _execute_service await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)( File "/config/custom_components/watchman/__init__.py", line 240, in async_handle_report await async_report_to_notification( File "/config/custom_components/watchman/__init__.py", line 422, in async_report_to_notification data = {} if service_data is None else json.loads(service_data) File "/usr/local/lib/python3.10/json/__init__.py", line 339, in loads raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not dict

kbrown01 · March 14, 2023, 6:53pm

If anyone would like a Mushroom-ized version of this, I created one in my install:

type: custom:stack-in-card
title: Possible Issues
cards:
  - type: custom:auto-entities
    card:
      type: grid
      columns: 1
      square: false
    card_param: cards
    filter:
      template: >-
        {%- for item in
        state_attr("sensor.watchman_missing_entities","entities") -%}
          {{
            {
              'type': "custom:mushroom-template-card",
              'entity': item.id,
              'primary': item.state + ' ' + item.id,
              'secondary': item.occurrences,
              'icon_color': "red" if item.state == "missing" else "orange" if item.state == "unavail" else "yellow",
              'icon': "mdi:cloud-alert" if item.state == "missing" else "mdi:cloud-off-outline" if item.state == "unavail" else "mdi:cloud-question"
              }
            }},
        {%- endfor -%}

Looks like this:

teskanoo · April 2, 2023, 10:28pm

I have managed to achieve this - a bit late but I hope it helps someone


# https://github.com/custom-cards/flex-table-card
- type: custom:flex-table-card
  title: 'Watchman Missing Entities'
  entities:
    include: sensor.watchman_missing_entities
  columns:
  - name: Entity
    align: left
    data: entities
    modify: x.id
  - name: State
    align: left
    data: entities
    modify: x.state
  - name: Occurrences
    align: left
    data: entities
    modify: x.occurrences

chinedu40 · April 4, 2023, 12:34pm

Is there a way to set up an automation that run only when one specific entity is marked as unavailable or missing? I’ve tried using the following as a conditional template “{{ ‘camera.netatmo_camera’ in state_attr(‘sensor.watchman_missing_entities’, ‘entities’) }}” but it is always false event when camera.netatmo_camera is listed under ID in state attributes

the.loquitur · May 23, 2023, 6:31am

Hi,

I tried installing The Watchman and recieved the below error:

    https://github.com/dummylabs/thewatchman is not a valid add-on repository

I’m running 2023.5.3

stevemann · May 23, 2023, 2:04pm

How did you install it? I have watchman and I didn’t need to add a repository to my add-on repositories.

From the github link you provided:

Quick start

Go to the “Integrations” section on HACS, click “Explore and download repositories” and search for “watchman”, then click “Download this repository with HACS”.

Restart Home Assistant, go to Settings->Devices and Services->Add Integration and select Watchman integration. If integration not found, try to empty your browser cache and reload page.

Go to Developer Tools → Services, type watchman and select Watchman: report service then press “Call Service” button. Check watchman_report.txt file in your configuration directory.

finity · May 24, 2023, 3:13am

Watchman is not an add-on. It’s an integration.

you install it thru HACS and not the add-on store. HACS does not provide add-ons, only integrations.

the.loquitur · May 24, 2023, 6:59am

Thanks ! I’m new to HA and didn’t have HACS installed. I just did so and was then able to get Watchman installed as well.

Ildar_Gabdullin · July 23, 2023, 6:38pm

Thank you, here is a bit modified version:

type: custom:flex-table-card
title: Watchman Missing Entities
entities:
  include: sensor.watchman_missing_entities
columns:
  - name: Entity
    data: entities
    modify: x.id
  - id: id_state
    name: State
    data: entities
    modify: x.state
  - name: Occurrences
    data: entities
    modify: x.occurrences.split(':')[0]
  - name: Lines
    data: entities
    modify: x.occurrences.split(':')[1]
sort_by: id_state

I myself prefer sorting by the “Occurrences” column:

    sort_by:
      - id_state
      - id_occurrences ##add this key to the corresponding column

Also, for me “Occurrence” is needed only for missing entities, then it may be done by this:

      - id: id_occurrences
        name: Occurrences
        data: entities
        modify: >-
          if(x.state == 'missing')
            x.occurrences.split(':')[0]
          else
            ''
      - name: Lines
        data: entities
        modify: >-
          if(x.state == 'missing')
            x.occurrences.split(':')[1]
          else
            ''

Also, for me it is better to have 3 separate lists for unavailable, unknown, missing; registered a FR for this.

A small remark: noticed a small assumption (absolutely not critical): only one “occurrence” is reported for an “entity”, but an “entity” may have several “occurrences” (i.e. several files). Absolutely unimportant: a user may “cleanup” missing entities in reported files, then “re-watch” and check other files from a new report.

KruseLuds · August 6, 2023, 1:06am

@scr, this is fantastic!

if my notification service is an email, what do I put in this section - just to show the whole report in the body of the email?

Thanks for any help in advance!

Edwin_D · September 30, 2023, 4:32pm

I’ve been moving some of the yaml based items to helpers, but now I start to wonder: Does Watchman also report on helpers with missing entities, for instance a Threshold helper, template helper, etc with missing entities?

Ildar_Gabdullin · September 30, 2023, 4:59pm

Depends on if Watchman monitors the “core.config_entries” file.
In my observations, Watchman does not see when a new shiny UI-based template sensors become unavailable.
Off-topic: for me, see zero reasons to move my setup to UI-based helpers / templates.

Edwin_D · September 30, 2023, 5:18pm

Well, to give an example: I use thresholds a lot (is it dark outside so lights should go on and stuff like that). From the UI the threshold limits are accessible via the gui and do not require a restart when limits are changed. Also, they get displayed with the device they are linked to, it represents there as an entity same as the original ones. This makes navigation to those entities easier. And they can be assigned to an area, while the config version could not because it didn’t allow for a unique id.

Ildar_Gabdullin · September 30, 2023, 5:23pm

Yes, there are SOME advantages of UI-based config - but there are disadvantages as well, and for me the latter is more important now.
Sorry for off-topic.

stevegroom · October 27, 2023, 3:36pm

Just looking at my Watchman integration
http://homeassistant.local:8123/hacs/repository/455174197
the graphics are missing.

stevegroom · October 27, 2023, 3:39pm

This is a great tool, it really comes in handy when I am trying to rationalise entity names.

Is there a way with manually or using watchman to know if any entities used in the ui are missing?
– I typically decide to rename an entity, run watchman and see what it finds and then have to manually check all the dashboard pages for entity missing errors.