VictoriaMetrics Add-on for long-term storage and data source for Grafana

madface · March 6, 2023, 6:57pm

The guys from victoriametrics sorted out the bug when importing empty measurements, now you get an errors but vmctl continues and everything works now.
They also helped me to find my imported data, maybe it helps someone other when switching.
Integrating victoriametrics like it is described here in the forum has a different naming of the measurements. For example, if you have a sensor like sensor.glances_cpu_used, it looks like this when ha saves the measurements in victoriametrics:

{"metric":{"__name__":"sensor.glances_cpu_used_value","db":"home_assistant","domain":"sensor","entity_id":"glances_cpu_used","friendly_name":"glances CPU used","unit_of_measurement":"%"}

So in fact you have a measurement for every entity_id. When using grafana or vmui you just have to enter your entity id and it will be found.
The imported data has always the name of the measurement, in the database it looks like this:

{"metric":{"__name__":"%_value","db":"homeassistant","domain":"sensor","entity_id":"glances_cpu_used"}

Every other entity that has a unit of measurement of “%” has also the __name__ "%_value". So to get the imported data you have to query

\%_value{entity_id="glances_cpu_used"}

Mind the backslash, if you have a special character in the beginning of the measurement you have to escape it with the \.

Zhao · March 16, 2023, 2:14pm

Can someone kind-hearted please share their influxdb config in configuration.yaml for VictoriaMetrics. Also, can you give me some info on the API syntax to get the last 100 readings from it before I get around trying to install Grafana etc?

I’m getting an error trying to integrate VictoriaMetrics with HA (docker),

2023-03-16 14:11:33.870 ERROR (SyncWorker_3) [homeassistant.components.influxdb] Cannot connect to InfluxDB due to 'HTTPSConnectionPool(host='localhost', port=8428): Max retries exceeded with url: /write?db=homeassistant (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)')))'. Please check that the provided connection details (host, port, etc.) are correct and that your InfluxDB server is running and accessible. Retrying in 60 seconds.

VictoriaMetrics is confirmed to be running at its metrics page.

hartmood · March 23, 2023, 5:33pm

I have been using the standard functionality for data visualization after switching from OpenHAB. By chance, I came across Victoriametrics. I have installed the VictoriaMetrics Addon and the configuration as stated in the documentation in the configuration.yaml. Now I do not get further:

Do I need the additional influxdb? Or is it already installed by default?
Will the data be stored in influxdb and the VictoriaMetrics DB?
Which host do I use in the configuration.yaml? Simply the IP address of my Raspbis? So with me 192.168.179.37
Is there anything else missing?

And a foolish question: How can I tell it is running? I appreciate any help you can provide.

Daniel15 · March 24, 2023, 6:01am

VictoriaMetrics has an API that’s compatible with InfluxDB. That means that anything that’s designed to use InfluxDB (for example, the Home Assistant InfluxDB integration) will work with VictoriaMetrics. It has a Prometheus-compatible API too.

You’d only need to run VictoriaMetrics, not InfluxDB.

Prometheus has (IMO) a better data model so it might be worth using the Prometheus Home Assistant integration instead of the InfluxDB one with VictoriaMetrics.

hartmood · March 24, 2023, 12:40pm

Thanks, So why is then a reference to the Influxdb in the configuration.yaml? I still struggle to understand if my setup is correct (host) and if the data is now directed to Victoriametrics. I also tried to find a tutorial on YouTube but only found one in French, which is unfortunately a little rusty after 35 years.

Daniel15 · March 26, 2023, 3:46am

Victoria Metrics has built-in support for clients that support InfluxDB - it can take InfluxDB-style data and convert it to VictoriaMetrics-style data. This means that apps do not need to support VictoriaMetrics specifically - they can just add support for InfluxDB or Prometheus and they’ll also work with VictoriaMetrics.

Home Assistant has native support for logging metrics to InfluxDB.

The InfluxDB section in configuration.yml is enabling Home Assistant’s InfluxDB support. Home Assistant thinks it’s speaking to an InfluxDB server, when it’s really speaking to a Victoria Metrics server.

Does that make sense?

Daniel15 · March 26, 2023, 4:20am

I was just trying out VictoriaMetrics (not via this addon, just via a regular Docker container) and Home Assistant’s metrics have much better structure if you use the Prometheus integration rather than the InfluxDB one.

You just need to add:

prometheus:

to your Home Assistant config, configure a prometheus.yml file as documented on VictoriaMetrics’ site, then pass it to VictoriaMetrics. This is what my docker-compose.yml looks like:

  victoriametrics:
    container_name: victoriametrics
    image: victoriametrics/victoria-metrics
    restart: unless-stopped
    environment:
     - TZ=America/Los_Angeles
    ports:
     - 8428:8428
    volumes:
     - /var/local/docker/victoriametrics/data:/victoria-metrics-data
     - /var/local/docker/victoriametrics/prometheus.yml:/etc/prometheus/prometheus.yml
    command:
     - "--promscrape.config=/etc/prometheus/prometheus.yml"
     - "-retentionPeriod=3y"

hartmood · March 29, 2023, 2:52pm

Thanks, Daniel. I followed the installation instructions, and it is running. At least I can see via Samba that data is constantly being written to the “small” folder, and I do have access via Grafana and a Prometheus connection:

With my below configuration.yaml is the data kept redundant? is influxDB installed with the setting, or do I have to install the add-on additionally?
With Prometheus as the source, I connected Grafana via http://ipaddress_of_raspi:8428, the user “home assistant”, and my password. What is the setting for configuring the InfluxDB source in Grafana? Can anybody provide a more detailed description of the configuration, as step 6 in the addon documentation could be more meaningful?
With the active Prometheus connection, how can I use Grafana dashboard templates that usually refer to InfluxDB? Is there a simple way to use them, or must I change the source in the JSON file?
Can I use templates for this setup not to reinvent the wheel? Where can I find them, and with what connection?

# Prometheus
prometheus:

# Victoria Metrics InfluxDB
influxdb:
  api_version: 1
  host: 192.168.179.37
  port: 8428
  max_retries: 3
  measurement_attr: entity_id
  tags_attributes:
    - friendly_name
    - unit_of_measurement
  ignore_attributes:
    - icon
    - source
    - options
    - editable
    - min
    - max
    - step
    - mode
    - marker_type
    - preset_modes
    - supported_features
    - supported_color_modes
    - effect_list
    - attribution
    - assumed_state
    - state_open
    - state_closed
    - writable
    - stateExtra
    - event
    - friendly_name
    - device_class
    - state_class
    - ip_address
    - device_file
    - unit_of_measurement
    - unitOfMeasure
  include:
    domains:
      - sensor
      - binary_sensor
      - light
      - switch
      - cover
      - climate
      - input_boolean
      - input_select
      - number
      - lock
      - weather
  exclude:
    entity_globs:
      - sensor.clock*
      - sensor.date*
      - sensor.glances*
      - sensor.time*
      - sensor.uptime*
      - sensor.dwd_weather_warnings_*
      - weather.weatherstation
      - binary_sensor.*_smartphone_*
      - sensor.*_smartphone_*
      - sensor.adguard_home_*
      - binary_sensor.*_internet_access

brackc · April 18, 2023, 10:50am

Thanks for the info on the docker-compose, it was very useful to help figure out the volume naming scheme.
N00b question: how and where would one run:

command:
- “–promscrape.config=/etc/prometheus/prometheus.yml”
- “-retentionPeriod=3y”

within the synology docker GUI when building the docker container?

Daniel15 · April 22, 2023, 1:41am

I’m not familiar with Synology’s Docker GUI, sorry! It’s probably worth asking in a Synology forum

brackc · May 20, 2023, 11:16pm

No worries, I’ll wait for the new Synology docker container manager to launch from beta and then build it from the .yml file info you set out above. I’ve still got some time before I have nay data in home assistant that I want to track long term (this is for when I eventually get solar to track usage patterns locally)

drahdiwaberl · July 3, 2023, 10:39pm

nevermind, it started to work today, after a reboot of the vm - might be that i had ‘upgraded’ it to 6GB of RAM (from 4), because i had restarted earlier yet without success…

~~i can’t get this addon started on homeassistant os running in a proxmox managed VM (i440fx)~~

~~log doesn’t seem to be much of help either:~~
</s> <s>s6-rc: info: service s6rc-oneshot-runner: starting</s> <s>s6-rc: info: service s6rc-oneshot-runner successfully started</s> <s>s6-rc: info: service fix-attrs: starting</s> <s>s6-rc: info: service fix-attrs successfully started</s> <s>s6-rc: info: service legacy-cont-init: starting</s> <s>s6-rc: info: service legacy-cont-init successfully started</s> <s>s6-rc: info: service legacy-services: starting</s> <s>s6-rc: info: service legacy-services successfully started</s> <s>[00:24:46] INFO: Starting Victoria Metrics with retention set to 99y</s> <s>[00:24:46] INFO: Starting Victoria Metrics with args set to </s> <s>2023-07-03T22:24:46.070Z info VictoriaMetrics/lib/logger/flag.go:12 build version: victoria-metrics-20230602-095850-tags-v1.91.2-0-gde9481208</s> <s>2023-07-03T22:24:46.070Z info VictoriaMetrics/lib/logger/flag.go:13 command-line flags</s> <s>2023-07-03T22:24:46.071Z info VictoriaMetrics/lib/logger/flag.go:20 -retentionPeriod="99y"</s> <s>2023-07-03T22:24:46.071Z info VictoriaMetrics/lib/logger/flag.go:20 -storageDataPath="/share/victoria-metrics-data"</s> <s>2023-07-03T22:24:46.071Z info VictoriaMetrics/app/victoria-metrics/main.go:70 starting VictoriaMetrics at ":8428"...</s> <s>2023-07-03T22:24:46.071Z info VictoriaMetrics/app/vmstorage/main.go:108 opening storage at "/share/victoria-metrics-data" with -retentionPeriod=99y</s> <s>2023-07-03T22:24:46.074Z info VictoriaMetrics/lib/memory/memory.go:42 limiting caches to 2466191769 bytes, leaving 1644127847 bytes to the OS according to -memory.allowedPercent=60</s> <s>2023-07-03T22:24:46.093Z info VictoriaMetrics/app/vmstorage/main.go:122 successfully opened storage "/share/victoria-metrics-data" in 0.022 seconds; partsCount: 0; blocksCount: 0; rowsCount: 0; sizeBytes: 0</s> <s>2023-07-03T22:24:46.096Z info VictoriaMetrics/app/vmselect/promql/rollup_result_cache.go:114 loading rollupResult cache from "/share/victoria-metrics-data/cache/rollupResult"...</s> <s>2023-07-03T22:24:46.097Z info VictoriaMetrics/app/vmselect/promql/rollup_result_cache.go:142 loaded rollupResult cache from "/share/victoria-metrics-data/cache/rollupResult" in 0.001 seconds; entriesCount: 0, sizeBytes: 0</s> <s>2023-07-03T22:24:46.097Z info VictoriaMetrics/app/victoria-metrics/main.go:80 started VictoriaMetrics in 0.026 seconds</s> <s>2023-07-03T22:24:46.097Z info VictoriaMetrics/lib/httpserver/httpserver.go:96 starting http server at http://127.0.0.1:8428/</s> <s>2023-07-03T22:24:46.097Z info VictoriaMetrics/lib/httpserver/httpserver.go:97 pprof handlers are exposed at http://127.0.0.1:8428/debug/pprof/</s> <s>

~~after that it’s still in stopped state~~

~~anyone know if and how this can be run in this environment?~~

~~tia.~~

dengolius · July 4, 2023, 8:46pm

VictoriaMetrics is like a prometheus but better.

dengolius · July 4, 2023, 8:48pm

We have ready-to-use docker containers and docker-compose files, see VictoriaMetrics/deployment/docker/docker-compose.yml at 6d35d21f60ef0ef2b05a7f9bde1a2feed636feb5 · VictoriaMetrics/VictoriaMetrics · GitHub

dengolius · July 4, 2023, 8:58pm

Use -selfScrapeInterval=10s flag for self scraping and VictoriaMetrics dashboard for Grafana to see memory/cpu consumption. It’s a good practice to use MoM(monitoring of monitoring) with configured vmalert and alerts: link 1 and link 2. See also Capacity planning and Resource usage limits.

Daniel15 · July 16, 2023, 11:47pm

Yes

What I meant in my comment is that Home Assistant has two integrations: an InfluxDB integration, and a Prometheus integration. I had a better result using the Prometheus integration with VictoriaMetrics, compared to using the InfluxDB integration. Prometheus has a nicer data model than InfluxDB.

PedroKTFC · July 17, 2023, 10:35pm

I’m a bit puzzled by the instructions. I’ve installed Victoria Metrics and the instructions then say:

Add influxdb integration to your homeassistant config (using the option measurement_attr: entity_id is recommended)

Firstly, there’s no such integration, if you try you get the message:

This device cannot be added from the UI
You can add this device by adding it to your ‘configuration.yaml’. See the documentation for more information.

However, there is an influxdb add on? Which should I use and how? Also, why do I need influxdb, isn’t Victoria Metrics its own database?

scadaguru · July 21, 2023, 11:26pm

Lots of good info here and I am thinking to switch to VictoriaMetrics from Influxdb.
I wonder someone can share prometheus.yml?
thanks

madface · July 22, 2023, 8:24am

@PedroKTFC

The addons are always the databases, you only need one of them (in this case the victoriametrics). But you need an integration that pushes the data to the addon, and for both it’s the influxdb integration.
The message you see in the ui means, this one cannot be setup from the ui, you have to manually add it to your configuration.yaml.

Here is an example, you find it also in the docs of the victoriametrics addon:

influxdb:
  api_version: 1
  host: <<<ADD-ON HOSTNAME FROM ADD-ON PAGE>>>
  port: 8428
  max_retries: 3
  measurement_attr: entity_id
  tags_attributes:
    - friendly_name
    - unit_of_measurement
  ignore_attributes:
    - icon
    - source
    - options
    - editable
    - min
    - max
    - step
    - mode
    - marker_type
    - preset_modes
    - supported_features
    - supported_color_modes
    - effect_list
    - attribution
    - assumed_state
    - state_open
    - state_closed
    - writable
    - stateExtra
    - event
    - friendly_name
    - device_class
    - state_class
    - ip_address
    - device_file
    - unit_of_measurement
    - unitOfMeasure
  include:
    domains:
      - sensor
      - binary_sensor
      - light
      - switch
      - cover
      - climate
      - input_boolean
      - input_select
      - number
      - lock
      - weather
  exclude:
    entity_globs:
      - sensor.clock*
      - sensor.date*
      - sensor.glances*
      - sensor.time*
      - sensor.uptime*
      - sensor.dwd_weather_warnings_*
      - weather.weatherstation
      - binary_sensor.*_smartphone_*
      - sensor.*_smartphone_*
      - sensor.adguard_home_*
      - binary_sensor.*_internet_access

I highly recommend not to include all your sensor but pick the ones (or domains) you want to have for a long term storage, so the database grows not so fast and you don’t store any garbage that you never need anymore in the future.

melwinek · August 16, 2023, 12:31pm

Is it possible to secure the VM with a login and password?