Script to help bulk extract ZHA router diagnostics files for AI analysis

Community Guides
, , , ,
1

I’ve been spending a lot of time trying to stabilise and improvey ZHA ZigBee network. With the help of AI tools like Gemini and Copilot, I’ve made massive progress on getting things running much better. The bash script I’ve shared was created through a few iterations in Gemini AI and allows you to quickly extract all your ZigBee Repeater/Router diagnostics logs and store them in a directory. Simply create a new file in the file editor (e.g. zha_bulk_extract.sh), paste in the script, update the long lived access token, chmod +X to the file to make it executable and then run the script via Terminal.

Once you’ve run the script it will save all your logs. Copy them to somewhere that is accessible to your favourite AI tool and then load them in and ask it to analyse the ZigBee router log files. I used Gemini to analyse 10 log files at a time and kept adding new prompts on the same session to analyse the 56 routers I had.
Provided your devices are well named with locations included in the names, AI can do wonders at drawing conclusions of what’s happening and provide rationale and recommendations.

#!/bin/bash

# --- CONFIGURATION ---
HA_URL="http://homeassistant:8123"
TOKEN="your long lived token here..-"
OUT_DIR="/config/zha_diagnostics_routers"
# ---------------------

if ! command -v jq &> /dev/null; then apk add jq; fi
mkdir -p "$OUT_DIR"

echo "1. Getting ZHA Entry ID..."
ENTRY_DATA=$(curl -s -f -H "Authorization: Bearer $TOKEN" "$HA_URL/api/config/config_entries/entry")
ZHA_ENTRY_ID=$(echo "$ENTRY_DATA" | jq -r '.[] | select(.domain == "zha") | .entry_id' | head -n 1)

if [ -z "$ZHA_ENTRY_ID" ] || [ "$ZHA_ENTRY_ID" == "null" ]; then
    echo "❌ FATAL: Cannot find ZHA ID. Check if ZHA is running."
    exit 1
fi
echo "   -> ZHA Integration ID: $ZHA_ENTRY_ID"

echo "2. Fetching Device List..."
TEMPLATE='{% for entity in integration_entities("zha") %}{{ device_id(entity) }}|{{ device_attr(device_id(entity), "name_by_user") or device_attr(device_id(entity), "name") }}{{ "\n" }}{% endfor %}'
PAYLOAD=$(jq -n --arg t "$TEMPLATE" '{template: $t}')

DEVICE_LIST=$(curl -s -X POST -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "$PAYLOAD" \
  "$HA_URL/api/template" | sort | uniq)

echo "3. Downloading & Filtering..."

count=0
skipped=0

while IFS='|' read -r dev_id dev_name; do
    if [ -z "${dev_id// }" ] || [ ${#dev_id} -lt 5 ]; then continue; fi

    safe_name=$(echo "$dev_name" | tr -dc '[:alnum:] ' | tr ' ' '_')
    filename="$OUT_DIR/${safe_name}_${dev_id}.json"
    url="$HA_URL/api/diagnostics/config_entry/$ZHA_ENTRY_ID/device/$dev_id"

    # Download
    curl -s -f -H "Authorization: Bearer $TOKEN" "$url" > "$filename"

    if [ ! -s "$filename" ]; then
        rm -f "$filename"
        continue
    fi

    # --- TYPE DETECTION ---
    # Recursive search for 'logical_type' anywhere in the file
    RAW_TYPE=$(jq -r '[.. | .logical_type? | select(. != null)] | .[0]' "$filename")

    # Filter Logic:
    # 0 = Coordinator (Keep)
    # 1 = Router (Keep)
    # 2 = EndDevice (Delete)
    
    if [[ "$RAW_TYPE" == "1" ]] || [[ "$RAW_TYPE" == "Router" ]] || [[ "$RAW_TYPE" == "router" ]]; then
        echo "   ✅ KEPT: $safe_name (Router)"
        ((count=count+1))
        
    elif [[ "$RAW_TYPE" == "0" ]] || [[ "$RAW_TYPE" == "Coordinator" ]] || [[ "$RAW_TYPE" == "coordinator" ]]; then
        echo "   ✅ KEPT: $safe_name (Coordinator)"
        ((count=count+1))
        
    elif [[ "$RAW_TYPE" == "2" ]] || [[ "$RAW_TYPE" == "EndDevice" ]] || [[ "$RAW_TYPE" == "end_device" ]]; then
        echo "   🗑️  Deleted: $safe_name (End Device)"
        rm -f "$filename"
        ((skipped=skipped+1))
        
    else
        echo "   ⚠️  UNKNOWN: $safe_name (Type: $RAW_TYPE) - Keeping."
        ((count=count+1))
    fi

done <<< "$DEVICE_LIST"

echo "---"
echo "Routers Saved: $count"
echo "End Devices Removed: $skipped"
1 Like
2

Based on the comprehensive analysis of all 5 batches of your router logs, here is the final summary of your Zigbee network health.
Network Health Score: B- (Strong Core, Fragile Edges)
Your network has a robust “backbone” driven by a powerful Coordinator and high-quality devices in the Kitchen/Master Bedroom. However, it suffers from significant “edge instability”—rooms at the far ends of the house (Rumpus, Ensuite) or devices with inferior radios (zbeacon) are failing to connect reliably.

  1. The “Danger Zones” (Priority Fixes)
    | Zone | Severity | Diagnosis | Recommended Fix |
    |—|—|—|—|
    | Rumpus Room | Critical | Network Depth Hazard. The “Rumpus Door Light” is 7 hops away from the coordinator. Zigbee becomes unstable after 4-5 hops. It is hanging by a thread. | Add a Repeater: Place a dedicated repeater (smart plug) physically halfway between the Kitchen and the Rumpus Room. |
    | Lounge / Living | Critical | The “Obstruction” Paradox. You have strong repeaters in this room (-69dBm), yet individual lights are failing (-90dBm). The “Mother of Pearl” lamp and “Hue Light 2” are likely blocked by furniture or metal lamp shades. | Move & Rotate: Pull the “Mother of Pearl” lamp away from the wall. Check if “Hue Light 2” is behind a TV or inside a metal enclosure. |
    | Study | High | Hardware Failure. The “Study Window Light” (zbeacon) is effectively dead (-89dBm) despite being in the same room as a perfect signal “Study Door Light” (-63dBm). | Replace Bulb: The “zbeacon” hardware is inferior. Replace it with a Philips, IKEA, or high-quality Tuya bulb. |
    | Ensuite / Bath | High | Signal Absorption. Tiles, pipes, and mirrors are blocking signals. RSSI is consistently weak (-84dBm). | Hallway Repeater: Install a smart plug in the hallway immediately outside the bathroom doors to “push” the signal in. |
    | Garage | Medium | Routing Error. The Garage Lamp is ignoring the strong nearby Smart Plug and trying to connect to a distant router. | Power Cycle: Turn off the Garage Lamp at the wall for 30 seconds to force it to find the nearby Smart Plug. |
  2. Hardware Performance Review
    Your logs provided a perfect A/B test of different brands operating in the same environment.
  • :trophy: Tier 1: The Anchors (Keep These)
    • Sonoff Zigbee Dongle Max (Coordinator): Performing perfectly.
    • Philips Hue & IKEA Tradfri: Consistently the strongest signals and most reliable repeaters.
    • Kitchen Power Board (Tuya): A surprisingly strong repeater. Keep this powered on at all times!
  • :warning: Tier 2: The “Hit or Miss”
    • eWeLink Bulbs: Some are great (Decking), others are terrible (Lounge). They seem very sensitive to physical obstruction.
    • Sonoff Dongle P (Routers): Decent, but ironically, your smart plugs are often outperforming these dedicated USB sticks.
  • :x: Tier 3: The Weak Links
    • zbeacon / Generic Tuya (Study Window): Consistently showed the worst signal reception relative to their location. Avoid buying more of these.
  1. Strategic Action Plan
  • The “Rumpus Rescue”:
    • You need to shorten the path to the Rumpus room. If you have a spare smart plug, place it in a room between the main house and the Rumpus room.
    • Heal the Network: After placing the plug, unplug your Sonoff Coordinator for 20 minutes, then plug it back in. This forces the Rumpus lights to recalculate their path (hopefully reducing Depth from 7 to ~4).
  • Fix the “Dead” Devices:
    • Lounge Zigbee Light (broken): Confirm if this is physically removed or just dead. Remove it from Home Assistant if gone to stop the network trying to find it.
    • Study Window Light: Replace this bulb. It is dragging down the mesh quality in that room.
  • Physical Checks:
    • Check the Pantry Light. Is it behind a metal door or surrounded by canned food? If so, it will never get a good signal unless a repeater is placed immediately outside the pantry door.
    • Check the Mother of Pearl Lamp. If the shell is thick or the base is metal, try raising the bulb higher or using a smart plug for the lamp instead of a smart bulb.
3

Also based on previous ZHA debugging log analysis, AI helped confirm the following:

  1. Tuya ZigBee power monitoring smart plugs spam the network to the point where it overloads it regardless of how good your other router network is. As a result I removed about 8 Tuya energy monitoring smart plugs from my network and reliability of my network increased significantly.
  2. Tuya air quality zigbee sensors spam the network effectively creating a denial of service attack with messages flooding 4 times per second. I removed mine from the network
  3. Tuya powered presence sensors also spam the network and I therefore removed the one I had. Battery powered Tuya presence sensors are less spammy so I kept mine on the network.
4

pretty interesting, while I am not much of a fan of ai in general (garbage data in, garbage data out), I do recognize they can be very good at pattern analysis.