HA requiring several reboots to see all Command line and File sensors

Hello!

All of my sensors have been shown in HA so config definitely works.

Sometimes on reboot of HA, several don’t show, requiring rebooting till they all show.
All of these sensors reference one file, tho some will appear and some not…

Any ideas?

What is your setup? How many sensors, and what kind are you having challenges with?

Hassio.
There are about 6 command line sensors referencing one file.
Only variance between the sensors in config.yaml is the reading reference the template is searching for. Ie: H1 or H2 etc.
Some sensors show. Some have exclamation marks where the sensors once were.
After a few boots they all eventually show.

The file they reference is inside the config file for HA and they are in config.yaml whitelist.

It sounds like it may be a timing or response issue from the commands. Have you looked at the error logs to see if there are any hints there?

A sample of config:

#   FILE and Command Sensors

#####################################################
  - platform: file
    name: Last Discharge
    file_path: /config/vedirect-bmv/bmv700.txt
    value_template: >
      {{ '%0.1f'|format((value | regex_findall_index("'H2': '(-*\d+)'") | int) / 1000) }}
    unit_of_measurement: 'Ah'
###############################################################
  - platform: command_line
    name: Avg Discharge
    scan_interval: 1
    command: 'tail -n 1 /config/vedirect-bmv/bmv700.txt'
    value_template: >
      {{ '%0.1f'|format((value | regex_findall_index("'H3': '(-*\d+)'") | int) / 1000) }}
    unit_of_measurement: 'Ah'
  ##############################################################################
  - platform: command_line
    name: Full Discharges
    command: 'tail -n 1 /config/vedirect-bmv/bmv700.txt'
    value_template: >
      {{ (value | regex_findall_index("\'H5\': \'(\d+)\'") | int) / 1 }}
    unit_of_measurement: 'Discharges'
#############################################################################

None in Developer Tools > Logs.

Is this file getting written onto the hassio config share by an external system? If so, can you check for errors/issues from the remote side?

It happens on random file and command sensors. Not just command :+1:

Checked the written file from Linux side.
No corruption.

It’s as if hassio is having problems extracting the values.

Sample of bmv700.txt

{‘FW’: ‘0308’, ‘SOC’: ‘936’, ‘Relay’: ‘OFF’, ‘PID’: ‘0x203’, ‘H10’: ‘2’, ‘BMV’: ‘700’, ‘TTG’: ‘-1’, ‘Alarm’: ‘OFF’, ‘H18’: ‘643’, ‘I’: ‘16491’, ‘H11’: ‘0’, ‘H12’: ‘0’, ‘CE’: ‘-18126’, ‘H17’: ‘486’, ‘P’: ‘232’, ‘AR’: ‘0’, ‘V’: ‘14049’, ‘H8’: ‘14148’, ‘H9’: ‘62440’, ‘H2’: ‘-62574’, ‘H3’: ‘-62393’, ‘H1’: ‘-66658’, ‘H6’: ‘-401763’, ‘H7’: ‘4’, ‘H4’: ‘3’, ‘H5’: ‘0’}

Very interesting.

I was wondering if there might be file locking problems, if the remote linux system is writing to the file at the same time that HA is trying to read the file. Possibly getting some inconsistent results.

Just speculation on my part.

1 Like

That sounds like a possibility!

Below is the python script I’m using to extract and write the data file (hassio references).

Is there any way to put a spaced interval on the returned data in that script, like every 10 seconds?

(I just downloaded this script from the Victron Open Source web page)

It does update really fast currently. The file is about 30mb per day!

I run this python script from crontab -e using the command below and use logrotate to keep the file size under control.

@reboot cd /usr/share/hassio/homeassistant/vedirect-bmv && python vedirect.py --port /dev/ttyUSB-BMV1 > bmv700.txt





#!/usr/bin/python
# -*- coding: utf-8 -*-

import os, serial, argparse

class vedirect:

    def __init__(self, serialport, timeout):
        self.serialport = serialport
        self.ser = serial.Serial(serialport, 19200, timeout=timeout)
        self.header1 = '\r'
        self.header2 = '\n'
        self.hexmarker = ':'
        self.delimiter = '\t'
        self.key = ''
        self.value = ''
        self.bytes_sum = 0;
        self.state = self.WAIT_HEADER
        self.dict = {}


    (HEX, WAIT_HEADER, IN_KEY, IN_VALUE, IN_CHECKSUM) = range(5)

    def input(self, byte):
        if byte == self.hexmarker and self.state != self.IN_CHECKSUM:
            self.state = self.HEX
            
        
        if self.state == self.WAIT_HEADER:
            self.bytes_sum += ord(byte)
            if byte == self.header1:
                self.state = self.WAIT_HEADER
            elif byte == self.header2:
                self.state = self.IN_KEY

            return None
        elif self.state == self.IN_KEY:
            self.bytes_sum += ord(byte)
            if byte == self.delimiter:
                if (self.key == 'Checksum'):
                    self.state = self.IN_CHECKSUM
                else:
                    self.state = self.IN_VALUE
            else:
                self.key += byte
            return None
        elif self.state == self.IN_VALUE:
            self.bytes_sum += ord(byte)
            if byte == self.header1:
                self.state = self.WAIT_HEADER
                self.dict[self.key] = self.value;
                self.key = '';
                self.value = '';
            else:
                self.value += byte
            return None
        elif self.state == self.IN_CHECKSUM:
            self.bytes_sum += ord(byte)
            self.key = ''
            self.value = ''
            self.state = self.WAIT_HEADER
            if (self.bytes_sum % 256 == 0):
                self.bytes_sum = 0
                return self.dict
            else:
                print 'Malformed packet'
                self.bytes_sum = 0
        elif self.state == self.HEX:
            self.bytes_sum = 0
            if byte == self.header2:
                self.state = self.WAIT_HEADER
        else:
            raise AssertionError()

    def read_data(self):
        while True:
            byte = self.ser.read(1)
            packet = self.input(byte)

    def read_data_single(self):
        while True:
            byte = self.ser.read(1)
            packet = self.input(byte)
            if (packet != None):
                return packet
            

    def read_data_callback(self, callbackFunction):
        while True:
            byte = self.ser.read(1)
            if byte:
                packet = self.input(byte)
                if (packet != None):
                    callbackFunction(packet)
            else:
                break


def print_data_callback(data):
    print data

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Process VE.Direct protocol')
    parser.add_argument('--port', help='Serial port')
    parser.add_argument('--timeout', help='Serial port read timeout', type=int, default='60')
    args = parser.parse_args()
    ve = vedirect(args.port, args.timeout)
    ve.read_data_callback(print_data_callback)
    #print(ve.read_data_single())

I’m def not an expert. I would try writing a script on the linux side to parse the bmv700.txt file and then write that output to the hassio share. That should be much shorter, and you could have it execute in crontab at some appropriate interval. I’m assuming you are monting the /config samba share from linux?

That will also be a much smaller file you are writing onto your hassio host and if you are writing a SD card, it will be a lot less impact to the drive life.

Parsing is something I’ll have to look into. The bmv700.txt file isnt json and I’ve no idea how to extract a complete:

{‘FW’: ‘0308’, ‘SOC’: ‘936’, ‘Relay’: ‘OFF’, ‘PID’: ‘0x203’, ‘H10’: ‘2’, ‘BMV’: ‘700’, ‘TTG’: ‘-1’, ‘Alarm’: ‘OFF’, ‘H18’: ‘643’, ‘I’: ‘16491’, ‘H11’: ‘0’, ‘H12’: ‘0’, ‘CE’: ‘-18126’, ‘H17’: ‘486’, ‘P’: ‘232’, ‘AR’: ‘0’, ‘V’: ‘14049’, ‘H8’: ‘14148’, ‘H9’: ‘62440’, ‘H2’: ‘-62574’, ‘H3’: ‘-62393’, ‘H1’: ‘-66658’, ‘H6’: ‘-401763’, ‘H7’: ‘4’, ‘H4’: ‘3’, ‘H5’: ‘0’}

String and write it to another file!

dap35 was right!

Solution continued here: