SSH'ing from a command line sensor or shell command

I’ve noticed there’s a significant number of posts looking for help on this topic and the posters all seem to struggle with a few common issues. I realized I’ve been almost copying and pasting my responses on these topics so it seemed like a good case for a community guide.

What will this cover

This guide will walk through the process of getting a command which ssh’s into another machine to get some simple data ready to be included in a command line sensor. To keep it simple I’ll use the same example from the documentation (getting CPU temperature) I’ll just run it over SSH. The focus will more be on how to test correctly, how to set up the identity file and how to handle the known hosts file in a way that works with HA.

Note that these instructions could also easily apply to any of the command line integrations or shell command. Also these instructions are going to focus on users that installed HA via the Home Assistant OS and Supervised install methods as this is where I’ve seen the most issues. It’s not useless in other install methods but it is going to talk about add-ons so those parts won’t be followable as is, you’ll have to adjust for your setup.

Pre-Requisites

  • The SSH & Web Terminal from the list of community add-ons set up and with protection mode disabled (this is necessary to be able to execute commands on specific containers, more on this below)

Portainer alternative
If you prefer to use the official SSH & Terminal addon or do not wish to disable protection mode on your SSH add-on then you can install the Portainer add-on add-on instead. You will still need to disable protection mode on the Portainer add-on (so we can see and run commands on other containers) but not on your SSH one in this case.

Generating the id file

Since we’ll be running these commands from sensors or service calls, interactive SSH isn’t an option so we must go password-less (plus its a security best practice). So to start we need to generate an identity file. The easiest way to do this is just to ssh into HA (or open “Terminal” from the left side nav of your UI) and then follow this guide (or if you don’t like mine, search for “setting up password-less SSH” and pick the guide of your choice).

However there is one key exception, do not store your key file in the default location. The default location for ssh-keygen is ~/.ssh which translates to /root/.ssh. You should not store anything in /root for a few reasons:

  1. It is not a shared folder between the docker containers. The SSH & Terminal add-on is actually a different docker container then the HA application which means they do not share a filesystem, they only share specific folders such as /config and /share. /root is not shared so anything you put in that folder while ssh’ed in like this is not visible to HA.
  2. Even if you do get to /root of the actual HA container, don’t use it. Only specific folders such as /config and /share are preserved across updates, everything else is wiped clean. So anything you do have stored in /root in the HA container will be lost next update.

Instead I’d recommend telling ssh-keygen to store its generated ID file in /config/.ssh. You don’t have to put it in a folder called .ssh if you don’t want to, I just like the consistency. But you should put it in /config so it is preserved over updates.

Testing your command

Now that you have your ID file, lets test the command. To do this though, we can’t just run the command while SSH’ed in normally because if your command works while ssh’ed into HA that does not mean it will work when run from HA via a command line sensor.

The reason for this is because of the docker containers like I mentioned before. When you SSH into HA you’re actually ssh’ing into an add-on which is a different docker container from HA itself. Each docker container has its own filesystem and installed packages, successfully running it while ssh’ed really doesn’t tell you anything about whether it will work when run from HA.

This is where disabling protection mode comes in. While SSH’ed into HA execute the following command (thanks @VDRainer!):

docker exec -it homeassistant bash

What this will do is open an interactive bash shell on the actual homeassistant container where HA runs. From this shell any commands we run will be run in exactly the same way HA will run them from a shell command or command line sensor. This way we know if they work from there then they will work from HA. So now we can run our command like so and confirm that it works:


Note that for those unaware, -i is the flag to the ssh command which tells it where to find the identity file to use. Since we have put the identity file in an unusual location we must specify this. Sub in the name and location you used for yours.

Portainer alternative

For those using the official SSH add-on or without protection mode disabled on their SSH add-on you won’t be able to use docker exec like I suggested above. Instead open the Portainer add-on from the left side nav of HA. From there click on ‘containers’
Screen Shot 2020-12-22 at 10.38.27 AM
Then find the ‘homeassistant’ container in the list and click the little icon for ‘exec console’ like this:


Then in this screen just click “connect” to launch a terminal window on this container as root:
Screen Shot 2020-12-22 at 10.42.39 AM
Now enter your command in the terminal that appears and run it like so:

Running it this way runs it in exactly the same way that HA does so now I have confirmed my command is working and can be put in a command line sensor.

Handling the known_hosts file

You’ll notice in my last screenshot that it asked me to verify the authenticity of the host. This is the final issue we have to tackle since we can’t have any interactive pieces in our final product.

Normally you just go through this once and then you are good to go, it never prompts you to verify authenticity again. However with HA this is another gotcha. The problem is that by default the known_hosts file is stored in /root/.ssh. Which means if you stop here your sensor will appear to work but will break next update when /root is wiped clean.

Fortunately, there’s a very simple solution to this, we just need to tell our command that the known_hosts file is somewhere that isn’t wiped clean every update. I would suggest also putting it within /config/.ssh but you can put it wherever you want, as long as its in somewhere like /config that is preserved over updates. Once you decide where to put it, add this to your command: -o UserKnownHostsFile=/config/.ssh/known_hosts. Then run your command once more from the docker exec shell with this addition (or from portainer if you are using the atlernative approach) so it updates your custom known_hosts file and then you are good to go:


Note: I used known_hosts_2 here because I already have a known_hosts file in this spot and I didn’t want to mess up my existing sensors by removing it. But again, you can call it whatever you want and put it wherever you want, as long as its somewhere in /config

Alternative - no host key check
There is an alternative you will see around the forum here. If you add -o StrictHostKeyChecking=no to your command then it will turn off host verification entirely. This works as well and is probably fine if you are going between two systems on your local network. But since this is a security bad practice I’m recommending the other way. In the end the decision is up to you though.

Command line sensor vs. Shell command

A few last things to be aware of. Shell commands runs in a restricted environment that doesn’t allow expanding the home director (~) or piping output of one command into another. This is the only situation I’ve personally run into so far where you have a command working in portainer that doesn’t work in HA. Make sure to read the documentation of the integrations you use to be aware of other integration specific gotchas like this.

Also if you are having trouble the command line integrations all log stdout and stderr if you set the log level of the integration you are trying to use to debug.

That’s it, hope it helps!

14 Likes

If you are looking for an example in practice, here’s one I made to keep my HA host up to date in my supervised install. There are plenty of others as well if you search around.

1 Like

Great, this is realy a guide we’ve been missing.
As an alternative, without portainer, it’s possible in the community addon-ssh to open a shell in the HA container if the protection mode is disabled (addon needs to be restarted).
Auswahl_418
Auswahl_419

2 Likes

Oh that’s a neat trick, I didn’t know that, thanks! Think I will update the guide in a bit to do that instead. Less screenshots of Portainer UI required and better to keep pre-reqs to one add-on :+1:

1 Like

Thanks, this is a very clear guide (also for ssh-dummies like me :slight_smile: )
It solved my problem in a couple of minutes.

1 Like

Unfortunately I don´t get there as I´m stuck at the very beginning:

➜  ~ ssh [email protected]
The authenticity of host '192.168.0.XXX (192.168.0.XXX)' can't be established.
ECDSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.0.XXX' (ECDSA) to the list of known hosts.
[email protected]: Permission denied (publickey).
➜  ~ ssh [email protected]
[email protected]: Permission denied (publickey).

So I can´t login with my remote user. I think that´s because that SSH server has already public key authentication enabled with one private key protected by a password. Priv key plus password are required on the client side.

Not sure how to proceed, adding that I realized this is not a HA but a pure SSH config issue. So:

  1. Do we really NEED password-less authentication on the remote SSH server? Or can I just add the private key plus password from the server to the client (HA) somehow?
  2. Would it be possible to add a new public/private key pair on the server where the private key is not protected by a password?

By the way “sshd_config” on the server/remote host has set

PermitRootLogin no
ChallengeResponseAuthentication no
PermitEmptyPasswords no
PasswordAuthentication no
RSAAuthentication yes
PubkeyAuthentication yes

Is that of interest?

SSH service status log of the remote host doesn´t show anything interesting, only:

Jun 22 22:43:32 192.168.0.XXX sshd[14575]: Connection closed by authenticating user remote 192.168.0.YYY port 48900 [preauth]

(where 192.168.0.YYY is the home assistant machine)

I wondered why on step 2 of SSH Passwordless Login Using SSH Keygen in 5 Easy Steps I wasn´t asked for the users password. That might be the root cause.

I really hope there´s a way to kickstart this, any guidance appreciated.

Sorry wait I’m a bit confused here, can you clarify what exactly is requiring a password? Because it looks like your sshd_config has PasswordAuthentication no. Are you being prompted for a private key passphrase or is this something else? You said you were being asked for a user password but I’m confused at how that could be given your sshd_config. Although its also possible its just beyond me, I’m a home user of SSH but not an IT expert so I definitely don’t know all the ins and outs here.

Maybe it would help if you showed what a successful SSH login looked like? Scrubbing out any private info of course.

Meanwhile I figured out it´s because of the PasswordAuthentication no. I temporarily changed it to yes and could access the remote host from Home Assistant.

So my task to accomplish (if possible) is to add the existing private key of the SSH remote host to Home Assistant, as password authentication is disabled for security reasons (and therefore I can´t go through the steps to enable password-less login).

It all comes back to :frowning:

Meanwhile I figured out it´s because of the PasswordAuthentication no. I temporarily changed it to yes and could access the remote host from Home Assistant.

So my task to accomplish (if possible) is to add the existing private key of the SSH remote host to Home Assistant, as password authentication is disabled for security reasons (and therefore I can´t go through the steps to enable password-less login).

It all comes back to… :frowning:

So probably not but its worth noting why this is difficult. Your commands must be non-interactive to work from shell commands and command line sensors and SSH doesn’t want you to use a password in non-interactive sessions for pretty valid reasons:

  1. It doesn’t want you to store the password in clear-text in a file (which you’ll have to do here)
  2. It doesn’t want anyone on the system to be able to see the password with a simple ps

That being said, if you’re comfortable with these risks I have an idea. With some searching I found that the go to tool for this task is one called sshpass. It accepts the password as a command line argument and uses it to start an SSH session as if it was entered interactively.

Now you can’t install anything in the Home Assistant container since any environment changes like that won’t be persistent. But sshpass is available as an alpine package which means you can add it to the list of alpine packages to install in the SSH & Web Terminal addon. Then what you should be able to do is essentially nest your SSH commands. First go password-less ssh to the addon then use sshpass to go from there to your remote server.

I want to be clear, I have not tested this as I use password-less SSH everywhere myself. But figured I’d throw it out there as it seems like it should work.

When you say server here, do you mean the remote machine? Cause I think that kind of depends on how it is configured? Like I certainly can, I simply generated pub/priv keys without a passphrase and that’s what I use. Is your server running software that requires a passphrase for private keys?

Interesting.

I´m currently one small step ahead: storing the private key of the existing server (remote host) public/private key pair in /config/.ssh/remote_private.txt and accessing the remote host from HA using “ssh -i /config/.ssh/remote_private.txt [email protected]” works. But it still asks for the password the private key is encrypted with.

I´ll try to provide a separate key pair on the remote host without a private key password, exclusively for the home assistant machine.

In the end we want to ssh into the remote host without the need of providing an additional password, right? If it´s okay to not just “ssh …” but “ssh -i /provide/path/to/private/key” for further usage that would be my favorite path to go. What do you think?

Question 1: Yes, remote machine.
Question 2: It´s a pretty well secured host (obviously) for good reasons. Therefore currently password for private key and PasswordAuthentication no set.

Update: Oh man… need more sleep. As I was sticking closely to SSH Passwordless Login Using SSH Keygen in 5 Easy Steps I always tried ssh [email protected] from the HA SSH Addon. But as stated later (a few steps later unfortunately…) in your post we need to use ssh [email protected] -i /config/.ssh/id_rsa !

So between the link here

and the first appearence of the -i parameter for ssh command here (even in a screenshot before)

the confusion happened for me.

Long story short: everything working as expected, my fault. Possibly the initial post would be bullet proof when noting to provide the custom path with the -i parameter when testing the SSH login. Standard links like the (basically good one) provided don´t fit the custom stuff we need to do here afterwards :smiley:

Additionally I used public/private keys a bit different over the years:
server → ONE key pair → use private key on all clients to access server

With the expansion of authorized_keys it´s possible to basically “vice versa whitelist” any client (its key). That approach was new to me, so learned a bit of basic Linux stuff before now turning to command line sensor and shell command fun with Home Assistant :slight_smile:

1 Like

I want to achieve the following:

  1. Create command_line sensor which connects to another docker container (running on the same host homeassistant container is running (HASS OS)) and gets some information

How can I do this? I used this guide to remotely access other SSH servers.
This time everything happens on the same host.

Can homeassistant containers SSH and root SSH be configured so homeassistant container can password-less login to root SSH (host SSH)? From a security point of view not the best way to go I guess.

command: "docker exec addon_a0d7b954_influxdb du -shm /data/influxdb/data/homeassistant | cut -f1"
→ This fails as it is run from HA in homeassistant container - where even no docker command exists. This command works on the root SSH very well.

Workflow I´m thinking about would look like:
HA container → SSH to host → login to another docker container → get information → done

(background: Unreliable InfluxDB size sensor - #34 by e-raser)

Oh wait, I see the problem. You said SSH so I assumed you had a container running an SSH service but that’s not it. You don’t actually want to SSH, you want to docker exec into a container.

As you noted the HA container does not have access to the docker CLI so that command won’t work from a commandline sensor. You will need to SSH from HA into somewhere else with docker CLI access and then execute that from there. Since it looks like you are running the InfluxDB add-on I would think that you have two options here:

  1. The workflow you outlined (SSH to host, run the docker exec command from there)
  2. Use the SSH & Web Terminal add-on (the community one) with protection mode turned off, that has access to the Docker CLI when you do that. Then use this workflow:
    • HA container → SSH to SSH & Web Terminal Add-on → login to another docker container → get information → done

Followed this example, but I am still getting:

Failed to add the host to the list of known hosts (/config/.ssh/known_hosts).

Did you create the /config/.ssh folder first? I tested just now and I am able to reproduce this error when the folder I want to put my known_hosts file in does not exist. As long as I make the folder first then it works without error.

That solved it. Wow, the .ssh folder permissions are really touchy! Thank you again.

1 Like