Anyway of reading JSON from config directory in Core

[kbrown01]

I have HA Core installed and the root directory is:

Path to configuration.yaml: /home/homeassistant/.homeassistant

In that root directory is a file that is created and modified by the Insteon integration. It is insteon_devices.json.

Now, I would like to create some template sensors, etc. that are based on this file. In order to do that, I really need to use REST because I need to be able to process the (huge) data using json_attributes_path. I say this because other sensors do not support json_attributes_path from what I see (like the command_line sensor which can read the file).

Is there anyway to read a file that is at that path (the root of the configuration)? I have tried many ways and it appears that it cannot be read. As a workaround, I have copied the file to something like:

/home/homeassistant/.homeassistant/www/insteon_devices

So I can access this with:

- platform: rest
  name: insteon_modem_aldb
  timeout: 30
  resource: http://192.168.2.245:8123/local/insteon_devices/insteon_devices.json
  value_template: "{{ value_json[0].address }}"
  json_attributes_path: "$.[0]"
  json_attributes:
      - aldb

But in reality, I would not want to access the copy. I would much rather access the actual file. I cannot find any address that allows this file to be grabbed via resource … like:

http://192.168.2.245:8123/insteon_devices.json

That yields a 404, File not Found. So is there a web address that corresponds to the root where configuration.yaml exists?

And by copying … yes I can create a command_line sensor that just copies the file I believe but that seems so wrong.

[123 (Taras)]

The resource option expects a REST endpoint (basically a web server able to reply to REST requests). You specified a file and that’s not a valid endpoint.

[kbrown01]

I do not understand what you said, this works perfectly:

http://192.168.2.245:8123/local/insteon_devices/insteon_devices.json

That is a file located at .homeassistant/www/insteon_devices/insteon_devices.json.

Translation – the web service that is /local on HA root is configured to serve files.

What I believe you are saying is that Home Assistant is not configured to serve files from that root configuration directory.

By the way, I can think of very good reasons why it doesn;t expose this directory, like someone could go after secrets.yaml. The issue that I have here is that the Insteon integration manages this file (like a cache) and I cannot change where it is. Perhaps the real solution is that this file in the integration should maybe be moved to somewhere else.

[kbrown01]

Alternatively I guess I could create another web service that allows access to that file or modify the existing web server in HA Core to provide access, but again that seems like hacks. Or maybe so pyscript which is the way I believe I may go that actually uses XSLT to read that JSON and modifies it, writing the result to another file that in turn is then read into a bunch of sensors.

[123 (Taras)]

@kbrown01

What I believe you are saying is that Home Assistant is not configured to serve files from that root configuration directory.

Exactly; it’s not a valid REST endpoint. You specified a file path but nothing at the destination will reply with the file’s contents.

As an experiment, add the following to configuration.yaml then restart Home Assistant.

homeassistant:
  allowlist_external_dirs:
    - "/config"

Your config file may already have a homeassistant key so just add the suggested information under it (i.e. don’t create duplicate homeassistant keys).

[kbrown01]

I tried that before. To be clear.
The .homeassistant directory does not have a /config directory.

If I add

  allowlist_external_dirs:
    - "/config"

HA will not run because there is no /config

If I instead use:

  allowlist_external_dirs:
    - "/"

HA starts but I cannot access the files. Possibly the question is … if I do allow “/”, what is the web service endpoint named? I did try adding it as in:

homeassistant:
  customize: !include customize.yaml
  media_dirs:
    music: /mnt/Music
  customize_domain:
    light:
      assumed_state: false
  allowlist_external_dirs:
    - "/config"

But as I said, HA will not even start because /config does not exist. I would believe the correct answer is “/” but still that does not work.

[bjeanes (Bo Jeanes)]

@kbrown01

But as I said, HA will not even start because /config does not exist. I would believe the correct answer is “/” but still that does not work.

Have you tried /home/homeassistant/.homeassistant in allowlist_external_dirs? I believe the /config value commonly shown is for those running Home Assistant OS / Docker which mounts that into /config in the core container (and add-on containers).

[gonzotek]

Couple of thoughts.

First and foremost - Putting the whole config directory up as web-accessible (even if only locally) is a TERRIBLE idea. Don’t do it under any circumstance. Don’t pursue this path. Just don’t do it. Seriously!

Now, you could symlink the file in the config directory to appear in www. I don’t know what all is in insteon_devices.json, but there’s a good chance you still wouldn’t want to expose it via a web address.

A better idea would be to use use a command line tool like jq or python’s json parser with command_line sensors. Either solution will take some extra work, but ultimately will be a secure way to access this info.

[kbrown01]

@bjeanes

/home/homeassistant/.homeassistant

Doesn;t work either. Thanks for the efforts. I think it would be better to just do some pyscript to copy and process the file. I tried many things like:

/home/homeassistant/.homeassistant
/.homeassistant
/

Cannot find anything/any path that allows access to that file.

[kbrown01]

Totally agree. As I said before I can see the exact reason no one would want to do this.
I think I will kill a few birds with that one stone … and do some script/json parser to process and write out files into my own directory. This will actually make it much easier to write out “good” and “targeted” JSON for my sensors.

[CentralCommand (Mike)]

I think something like this would work. No reason to copy it somewhere and try and go over REST, just parse it with jq. It’s better then json_attributes_path anyway.

sensor:
- platform: command_line
  name: Insteon modem aldb
  command: "jq '.[0]' insteon_devices.json"
  value_template: "{{ value_json[0].address }}"
  json_attributes:
    - aldb

Although I guess one note is make sure you have jq installed. Since you’re on a core install you’ll need to do that manually.

[kbrown01]

Thanks @CentralCommand, this looks like what could work with what I have. Just need to look into installing “jq”

[gonzotek]

What’s the OS? Raspbian/RaspberryPiOS, and most distros, should be as simple as installing from the native package manager.

[kbrown01]

It Linux Mint and I just checked with

sudo apt-get install jq

Already installed. So onto trying it out now!

[kbrown01]

@CentralCommand solution is perfect.

- platform: command_line
  name: Testing ALDB
  command: "jq '.[0]' insteon_devices.json"
  value_template: "{{ value_json[0].address }}"
  json_attributes:
    - aldb

[123 (Taras)]

@kbrown01

The .homeassistant directory does not have a /config directory

“config”, like “local”, isn’t the literal name of an existing directory. It refers to the Home Assistant’s root directory whose actual location depends on which installation method was used.

[kbrown01]

Exactly. I understand that … really … I was trying to see if there is one that references the root or there is a way to reference it. But as stated (even by me) that would be a huge no-no. The “jq” solution as a command line sensor works perfect and I have already integrated it in 10 secs.

[123 (Taras)]

@kbrown01

“jq” solution as a command line sensor works

True but only because you’re using Home Assistant Supervised (on an unsupported distro of Linux but that’s another discussion or maybe you’re using Home Assistant Container or Core). The majority of users use Home Assistant OS which doesn’t permit installing additional applications (like jq). Not that it matters for you but for most users the Solution can’t be implemented. EDIT I was wrong; command-line sensor executes in the container context, not the OS, so this will work (because jq is preinstalled in the homeassistant container). See CentralCommand’s post below.

[kbrown01]

Which is exactly why I use Core. Not that should be a matter of discussion in this thread but the title clearly states “Core”. Given that I have a big-ass server in my home closet with many things like Kodi and many other things, the need for a dedicated little box for me is not important.

Like 5TB storage
50,000 songs
Other apps that run in Linux for the house …

[CentralCommand (Mike)]

Actually at first I omitted this, it was only once I realized they were on core that I had to add it:

@CentralCommand

Although I guess one note is make sure you have jq installed. Since you’re on a core install you’ll need to do that manually.

jq comes preinstalled in any Home Assistant container. So on any other installation method that sensor just works, there’s no additional configuration required.