GIT Pull Addon

ScottSuine · March 7, 2019, 11:09am

Trying to get this add on to work, but received to following the log

Check SSH connection
[Warn] No valid SSH connection for [email protected]
[Info] Start adding SSH key
[Info] Setup deployment_key on id_rsa
[Info] setting up credential.helper for user: scott_suine
[Info] Saving git credentials to /tmp/git-credentials
fatal: could not read Username: No such device or address
[Info] Local git repository exists
[Info] Git origin is correctly set to [email protected]:ne/hassio.git
[Info] Start git fetch...
Warning: Permanently added the RSA host key for IP address '18.205.93.0' to the list of known hosts.
Load key "/root/.ssh/id_rsa": invalid format
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
[Error] Git fetch failed

I have a repo in my config directory which sucessfully pulls from the remote repository ( bitbucket) if I type git pull inthe cli, so I know SSH keys are set up correctly

J85 · March 17, 2019, 5:23am

I cant figure out the config either.

Codec303 · March 26, 2019, 9:04am

I’m using it on bitbucket without issue, however I’m not using an SSH key, I’ve put in my username and password in the config and it does a ‘git pull’ OK

nalipaz · April 1, 2019, 10:15pm

It took a bit for me to get this working but I did. I had to do a few things.

Installed and configured the ssh addon.
SSH’ed to the hass.io install
Took a backup of the /config/ directory by creating a tarball
exited ssh and downloaded my tarball using scp
Installed and configured the git pull addon using a config similar to (could not get the rsa key to work, only a username/password [used a personal access token]):

{
  "deployment_key": [],
  "deployment_key_protocol": "rsa",
  "deployment_user": "username",
  "deployment_password": "redacted",
  "git_branch": "master",
  "git_command": "pull",
  "git_remote": "origin",
  "git_prune": false,
  "repository": "https://github.com/username/repo.git",
  "auto_restart": false,
  "restart_ignore": [
    "ui-lovelace.yaml",
    ".HA_VERSION",
    ".cloud/",
    ".gitignore",
    ".storage/",
    "OZW_Log.txt",
    "deps/",
    "home-assistant.log",
    "home-assistant_v2.db",
    "options.xml",
    "pyozw.sqlite",
    "tts",
    "zwcfg_0xc8ca1698.xml",
    "zwscene.xml"
  ],
  "repeat": {
    "active": true,
    "interval": 300
  }
}

Saved the config
Started git pull addon and set it to auto-start
Did a comparison of my downloaded tarball files and the /config/ directory now that git pull addon is enabled
Copied all the missing files from my prior backup of that directory into /config/. Things like zwave config files and home-assistant database file
Once everything was all matched up I rebooted hass.io from the UI and everything is good

The thing that was difficult for me to grasp and understand was that the addon completely wipes out everything prior to install, including zwave config, discovered components, and all the automagically created stuff that is stored in other config and database. Once I realized that I was able to circumvent the issue by copying over all the files that initial process creates.

Guff666 · August 11, 2019, 12:08pm

I want to use this addon in the following workflow:

Make changes to the local files using Atom and the samba addon
Push the changes to github for backup and travis-ci testing
Let the ‘Git Pull’ addon pull the tested changes back and restart hass.

Unfortunately, I can’t get ‘git pull’ to run properly. I suspect for two reasons:

1 The github remote is not ‘origin’, it’s ‘github’ (I use origin for a local gitosis git server)
2 I use the git protocol to access github and not https://

I get an error when I run this

[Error] git origin does not match git://[email protected]:garethhowell/Home-Assistant-Configuration.git!

I do specify that

"git_remote": "github",

but I wonder if this is being ignored because it’s always been used with

  "git_remote": "origin",

Gareth

agners · August 28, 2019, 9:08pm

I also did have problems configuring the add-on at first. Copying the example config and go from there helped me. I think it was the mandatory .git ending. Btw, this restrictions does not make sense since .git is not necessary if you create a bare repository and access it via ssh.

Lifelogger · January 8, 2020, 7:13pm

Anyone here who got the plugin working recently?

I can do a manual pull and push, but I don’t want to have to enter my username and password all the time. So I would like to do it with the plugin.

However, I keep getting the error:

fatal: could not read Username: No such device or address

and

git origin does not match https://github.com/username/repo.git

Any help is welcome! Replace the real username and repo btw.

Codec303 · January 9, 2020, 1:20am

I can do a manual pull and push, but I don’t want to have to enter my username and password all the time. So I would like to do it with the plugin.

If you can already do a manual pull/push then you may just need to set up an SSH key do you don’t need a username / password every time.

Lifelogger · January 9, 2020, 7:11am

@Codec303 That is actually a good idea. Haven’t thought about that. Thanks. Will look into that.

mdrichardson · January 23, 2020, 10:29pm

In case anybody else has trouble using rsa/ssh keys, I had to:

Follow the GitHub SSH setup
git remote set-url origin "[email protected]:<yourUserName>/hassio.git"
In the addon config, set respository to [email protected]:<yourUserName>/hassio.git
No username or password…just add your private key to deployment_key

florck · February 10, 2020, 10:37am

Hi.

I successfully installed the plugin and initiated it. This is very powerfull. However I have encountered a bug. The plugin stop accepting the rsa key after few hours :

Check SSH connection
[Warn] No valid SSH connection for ssh
[Info] Start adding SSH key
[Info] Setup deployment_key on id_rsa
[Info] Local git repository exists
[Info] Git origin is correctly set to ssh://[email protected]:2222/home-assistant/home-flamel.git
[Info] Start git fetch...
Load key "/root/.ssh/id_rsa": invalid format

[email protected]: Permission denied (publickey).

fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
[Error] Git fetch failed

I need to restart the plugin to get it back to work:

Check SSH connection
[Warn] No valid SSH connection for ssh
[Info] Start adding SSH key
[Info] Setup deployment_key on id_rsa
[Info] Local git repository exists
[Info] Git origin is correctly set to ssh://[email protected]:2222/home-assistant/home-flamel.git
[Info] Start git fetch...
Warning: Permanently added '[git.afola.ovh]:2222,[52.49.89.82]:2222' (ECDSA) to the list of known hosts.

[Info] Staying on currently checked out branch: master...
[Info] Start git pull...
Already up to date.
[Info] Checking if something has changed...
[Info] Nothing has changed.

My config is:

deployment_key:
  - '-----BEGIN OPENSSH PRIVATE KEY-----'
  - HIDDEN
  - '-----END OPENSSH PRIVATE KEY-----'
deployment_key_protocol: rsa
deployment_user: ''
deployment_password: ''
git_branch: master
git_command: pull
git_remote: origin
git_prune: false
repository: 'ssh://[email protected]:2222/home-assistant/home-flamel.git'
auto_restart: true
restart_ignore:
  - ui-lovelace.yaml
  - .gitignore
  - appdaemon/
  - .flake8
  - Pipfile
repeat:
  active: true
  interval: 60

Have you experienced this issue on your side ?

chickenandporn · April 30, 2020, 4:40pm

Agreed – I do a pre-check in a Jenkins container, then commit to master; in past workflow (hass, icinga/nagios) I’ve pushed to a remote repos that acts immediately to validate and update, but sits idle without polling in the meantime.

I would want to re-use code, and I do have a PR out for a deploy_key error in git_pull, it might be worth extending to make it accept pushes? Or would that cloud the intention of git_pull too much ?

chickenandporn · April 30, 2020, 4:43pm

I have experienced this and fixed it. There is a PR at https://github.com/home-assistant/hassio-addons/pull/1218

Feel free to comment in it.

Root cause: your ~root/.ssh/id_* file is constantly growing, and becomes too large.

Migitate: completely delete that key file, it gets recreated from the JSON config each pull. Every 5 min by default.

hrdejong · October 5, 2020, 5:16pm

Anyone try to use a .gitignore file to leave stuff alone? Doens’t seem to work… so seems useless to me.

mainmachine · February 6, 2021, 3:56pm

It does work, but I’ve found .gitignore regex syntax throws me off because it’s ever so slightly different from bash syntax for some things. I don’t recall where I got this but it was specifically recommended for use with this addon:

# Example .gitignore file for your config dir.
# An * ensures that everything will be ignored.
*

# You can whitelist files/folders with !, these will not be ignored.
!*.yaml
!.gitignore
!*.md
!esphome/
!themes/
!light_profiles.csv
!*.h

# Ignore folders.
.storage
.cloud
.google.token

# Ensure these YAML files are ignored, otherwise your secret data/credentials will leak.
ip_bans.yaml
secrets.yaml
known_devices.yaml

This approach ignores everything by default, so it’s a conservative approach and works by making exceptions for what you want to track.

torinmb · March 24, 2021, 1:08am

I think there’s a bug using an RSA key that’s generated with the ed25519 protocol even though the deployment_key_protocol accepts that format. I had to generate one with the legacy rsa format based on github’s instructions in /root/.ssh/
specifically:

$ ssh-keygen -t rsa -b 4096 -C "[email protected]"

I then had to copy the contents of the id_rsa into the Git Pull deployment_key

Also I had to use the Terminal & SSH component and in the /config folder I ran

git init.

In my case, I had already copied the specific files I wanted to track/update into my git repo and at that point wanted to stash any local changes I had made on my pi, so I ran the following:

git add .
git stash

At this point I could start the Git Pull component using the following configuration in the add on:

deployment_key:
  - '-----BEGIN OPENSSH PRIVATE KEY-----'
  - removed (note the key here didn't need quotes)
  - '-----END OPENSSH PRIVATE KEY-----'
deployment_key_protocol: rsa
deployment_user: ''
deployment_password: ''
git_branch: master
git_command: pull
git_remote: origin
git_prune: false
repository: '[email protected]:username/repo.git'
auto_restart: false
restart_ignore:
  - ui-lovelace.yaml
  - .gitignore
repeat:
  active: false
  interval: 300

providing a username + password, or username + security key didn’t work for me.
Also, the repository had to be in an ssh format instead of an https format.

torinmb · March 25, 2021, 2:02am

Has anyone figured out how to prevent git pull from initially wiping the PI? I have multiple devices and I’ve set them up to pull the same automation and scripts, but they each need their own cloud config and other settings, so I can’t back up the entire PI in the Git Repo. This plugin works great after the initial wipe, but because of the initial data loss I can’t set it up on more devices.

Any ideas on how to prevent the initial wipe would be amazing! Thanks

Tuxinator94 · September 11, 2021, 1:44pm

Is it possible to specify a file name pattern in the restart_ignore list? For example, I want to exclude any files ending with .sample or .md from triggering a restart?

I tried:

restart_ignore:
  - *.md

but get an invalid yaml error and it won’t let me save the config.

dunxd · June 5, 2022, 9:59pm

I am trying to set this up, but struggling to get it to use my ssh key. It is not at all clear how to define the key. There is a field in the config that allows entry of the key, but copying and pasting it from the id_rsa field into the field doesn’t seem to work. Looking at the docs, it seems there may have been an odd format for adding this into the YAML config, and this is made more confusing by the switch to GUI config and no way to see the underlying YAML [that I can figure out].

I get the following error message in logs:

[Info] Start git fetch...
Load key "/root/.ssh/id_rsa": invalid format
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

I am successfully using the same SSH key with the File Editor add-on, so I know the problem isn’t with the key or the repository. I do recall that the File Editor add-on couldn’t read the ssh key from /root/.ssh - I had to copy it to /config/.ssh. However, this Git Pull add-on requires manual entry of the whole key, so I am confused as to why it refers to an id_rsa file.

Has anyone got this working using the current config method?

dunxd · June 5, 2022, 10:29pm

Ok - I’ll answer my own question.

Use the … menu at the top of the configuration page to select Edit in YAML. If you already pasted in a long SSH key, then you will need to scroll right to see the … menu! That’s why I missed it.

Put the --- BEGIN OPENSSH PRIVATE KEY --- line in single quotes.
Put the key itself into it’s own line using >- to make it multiline. The relevant part of the config looks something like this:
Put the --- END OPENSSH PRIVATE KEY --- line in single quotes, and on a new line.

deployment_key:
  - '-----BEGIN OPENSSH PRIVATE KEY-----'
  - >-
    blahrandom
    anotherlineofrandom
    yetanotherlineofrandom
  - '-----END OPENSSH PRIVATE KEY-----'

Restart the add-on and it will start working.