I’m trying to sync my config files to GitHub but I’m stuck on the pull command.
I’ve added only the configuration.yaml file to the commit, so it should be supper small, but when I try to commit it enumerates 2078 objects and fail when trying to transfer more than 2Gb
My configuration.yaml is quite small, so I believe this is more related to previous work, but as I’m not familiar with GitHub I couldn’t get out of this.
➜ /config git:(main) ✗ git push origin main
Enumerating objects: 2078, done.
Counting objects: 100% (2078/2078), done.
Delta compression using up to 4 threads
Compressing objects: 100% (1997/1997), done.
error: RPC failed; curl 55 Failed sending data to the peer
send-pack: unexpected disconnect while reading sideband packet
Writing objects: 100% (2078/2078), 2.32 GiB | 10.28 MiB/s, done.
Total 2078 (delta 513), reused 3 (delta 0), pack-reused 0
fatal: the remote end hung up unexpectedly
Everything up-to-date
I don’t really remember the specifics of what I setup but I have daily backups to GitHub automated.
It uploads everything except what is excluded using gitignore so my guess is yours is indeed trying to upload your entire directory including a large database.
You will be asked to enter your GitHub username and password
Should this be updated now that GitHub no longer accepts password authentication? As it stands, it’s waiting to trip up those following a ‘how to’ guide?
Hello everyone. I had setup github upload script to upload only certain files into a private repo. I got an error yesterday saying the log file is too big. The thing is I am excluding any kind of logs to upload to github and yet I still get this
Warning: Permanently added 'github.com' (EDxxxxx) to the list of known hosts.
Enumerating objects: 445, done.
Counting objects: 100% (445/445), done.
Delta compression using up to 4 threads
Compressing objects: 100% (434/434), done.
Writing objects: 100% (445/445), 17.34 MiB | 363.00 KiB/s, done.
Total 445 (delta 283), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (283/283), done.
remote: warning: File home-assistant.log.1 is 61.24 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 68.25 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 62.71 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 61.87 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 59.40 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 51.19 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 60.31 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 60.87 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 60.77 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File home-assistant.log.1 is 63.90 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: error: Trace: 3d3b1566d4659df1f430b6becf8947deeb55bef6eb7ff5f87dd812c1c282f706
remote: error: See https://gh.io/lfs for more information.
remote: error: File home-assistant.log.1 is 165.87 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
To github.com:username/myrepo.git'
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'github.com:username/myrepo.git'
One possibility?: Your .gitignore file tells git which files to ignore in the future, but if you have added them in the past manually, or if they were added before .gitignore exluded them, they will still be tracked, and you’ll need to exclude them explicitly. (E.g. with git rm).
Also, I’d probably put something like *.log* in the .gitignore , rather than listing individual files.
Adding this as much for my notes as anything else.
For anyone else struggling to convert from the old username/password to the new tokens, the answer is to remove and re-add the repository, using the git command line. This post spells out that part.
First, add a new personal access token in Github, I used these permissions:
Then, log into HA using the SSH addon.
At the command line in the config directory (the default), find your repo name with git remote -v
You should get something like origin [email protected]:YourUsername/YourRepositoryName.git (fetch) origin [email protected]:YourUsername/YourRepositoryName.git (push)
Remove it with git remote remove origin
…and add it back with: git remote add origin https://github.com/YourUsername/YourRepositoryName
Finally, trigger a push with git push -u origin master
You should now be prompted for credentials; use your usual username, and for the password enter the new token.
Thanks to all. Anyone know if is it possible to make an automation (or schedule a cronjob) to automate git push?
In other words, how can pass username and token to git?
I think I’ve followed everything but am clearly missing something:
I’ve created my repository
I’ve set up SSK key
I’ve added the key to GitHub
…but I still need a passphrase to enter ilo the old password when pushing and I have no clue where/what that is. There was no prompting to enter a passphrase when setting up the SSH key.
Hi,
prefered the whitelist approach. Also have node-red with a lot of config there. I’ve extended it to additional modules I have. Some store static data in the /config directory so a little bit of trial was needed. I’m sharing in case someone would like to reuse.
# Home Assistant .gitignore - Comprehensive Whitelist Approach
# Deny everything by default, then explicitly allow configuration files
# Deny everything first
*
!*/
# Track this file
!.gitignore
# Core configuration files
!config/configuration.yaml
!config/automations.yaml
!config/scripts.yaml
!config/scenes.yaml
!config/known_devices.yaml
# Domain-specific configuration files (include all that might be present)
!config/climate.yaml
!config/cover.yaml
!config/group.yaml
!config/input_boolean.yaml
!config/input_button.yaml
!config/input_datetime.yaml
!config/input_number.yaml
!config/input_select.yaml
!config/input_text.yaml
!config/light.yaml
!config/media_player.yaml
!config/switch.yaml
!config/sensor.yaml
!config/binary_sensor.yaml
!config/camera.yaml
!config/device_tracker.yaml
!config/notify.yaml
!config/zone.yaml
!config/person.yaml
!config/recorder.yaml
!config/history.yaml
!config/logbook.yaml
!config/logger.yaml
!config/customize.yaml
!config/customize_domain.yaml
!config/customize_glob.yaml
# Packages directory
!config/packages/
!config/packages/**/*.yaml
!config/packages/**/*.yml
# Blueprints
!config/blueprints/
!config/blueprints/**/*.yaml
!config/blueprints/**/*.yml
# Packages
!config/packages/
!config/packages/**/*.yaml
!config/packages/**/*.yml
# AppDaemon
!config/appdaemon/
!config/appdaemon/appdaemon.yaml
!config/appdaemon/apps/
!config/appdaemon/apps/**/*.py
!config/appdaemon/apps/**/*.yaml
!config/appdaemon/apps/**/*.yml
!config/appdaemon/dashboards/
!config/appdaemon/dashboards/**/*.dash
# Exclude AppDaemon runtime files
config/appdaemon/compiled/
config/appdaemon/namespaces/
config/appdaemon/www/
# Node-RED (if used)
!config/node-red/
!config/node-red/**/*.json
!config/node-red/**/*.js
config/node-red/**/node_modules/
# Zigbee2MQTT
!config/zigbee2mqtt/
!config/zigbee2mqtt/**/*.yaml
!config/zigbee2mqtt/**/*.yml
!config/zigbee2mqtt/**/*.json
# ESPresense
!config/espresense/
!config/espresense/**/*.yaml
!config/espresense/**/*.yml
# MQTT discovery or other integrations
!config/mqtt/
!config/mqtt/**/*.yaml
!config/mqtt/**/*.yml
# Lovelace UI configuration files (if not stored in .storage)
!config/ui-lovelace.yaml
!config/lovelace/
!config/lovelace/**/*.yaml
!config/lovelace/**/*.yml
# Themes
!config/themes/
!config/themes/**/*.yaml
!config/themes/**/*.yml
# Python scripts
!config/python_scripts/
!config/python_scripts/**/*.py
# Shell commands and scripts
!config/shell_commands/
!config/shell_commands/**/*
# WWW directory (static files for Lovelace/frontend)
!config/www/
!config/www/**/*.js
!config/www/**/*.css
!config/www/**/*.html
!config/www/**/*.json
!config/www/**/*.yaml
!config/www/**/*.yml
# Allow additional file types for custom cards/resources
!config/www/**/*.ts
!config/www/**/*.map
!config/www/**/*.LICENSE.txt
# Exclude certain auto-generated or large files in www
config/www/**/*.gz
# Exclude frigate as it stores config in lovelace
config/www/community/frigate-hass-card/*
# Exclude advanced-camera-card
config/www/community/advanced-camera-card/*
# Version tracking
!config/.HA_VERSION
# Allow documentation files only in root
!*.md
/config/**/*.md
# Addon configurations - Allow essential config files only
addon_configs/**/*
!addon_configs/*/
# Node-RED addon configs
!addon_configs/*/flows.json
!addon_configs/*/package.json
!addon_configs/*/settings.js
!addon_configs/*/.config.nodes.json
!addon_configs/*/.config.runtime.json
!addon_configs/*/.config.users.json
!addon_configs/*/node-red-contrib-*.json
# Exclude Node-RED runtime files
addon_configs/*/node_modules/
addon_configs/*/.flows.json.backup
addon_configs/*/context/
addon_configs/*/nodes/
addon_configs/*/lib/
addon_configs/*/.config*.backup
addon_configs/*/.npm/
# Exclude all runtime data and cache directories
config/.storage/
config/.cloud/
config/deps/
config/tts/
**/__pycache__/
**/*.pyc
**/*.pyo
**/*.pyd
# Exclude database and log files
**/*.db
**/*.db-shm
**/*.db-wal
**/*.log
**/*.log.*
**/*.pid
**/*.sock
**/*.tmp
**/*.backup
**/*.bak
# Exclude Home Assistant specific runtime directories
config/components/
config/.uuid
config/.HA_VERSION.*
# Exclude large media files that shouldn't be in Git
**/*.mp4
**/*.avi
**/*.mkv
**/*.mov
**/*.mp3
**/*.wav
**/*.flac
# Exclude certificate and key files (security)
**/*.pem
**/*.key
**/*.crt
**/*.p12
**/*.pfx
# Exclude service account and credential files
**/SERVICE_ACCOUNT.JSON
**/client_secret*.json
**/google_assistant_*.json
**/*_credentials.json
# Exclude secrets file (move sensitive data here)
config/secrets.yaml
# OS specific files
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
#!/usr/bin/env sh
NOW=$(date +"%d/%m/%Y %H:%M")
echo "${NOW} Git Update" > /config/logs/git.log 2>&1
# set's the WDIR to the current directory (location of the script)
WDIR=$(cd `dirname $0` && pwd)
ROOT=$(dirname ${WDIR})
# Update fake_secrets.yaml
#echo "Updating fake_secrets.yaml"
if [ -x "${WDIR}/make_fake_secrets.sh" ]; then
"${WDIR}/make_fake_secrets.sh"
else
echo "Error: make_fake_secrets.sh not found or not executable"
exit 1
fi
# Ensure github.com is in the known_hosts files
# this is required as every upgrade of HA resets this
KNOWN_HOSTS_FILE="/root/.ssh/known_hosts"
HOST="github.com"
# Check if the host is already in known_hosts
if ! grep -q "$HOST" "$KNOWN_HOSTS_FILE"; then
echo "Adding $HOST to known_hosts..."
ssh-keyscan "$HOST" >> "$KNOWN_HOSTS_FILE"
else
echo "$HOST already present in known_hosts."
fi
# Git config
git config --global user.name xxxxxxx
git config --global user.email [email protected]
# Add new files
git add . >> /config/logs/git.log 2>&1
echo "-----> git add done"
git status
echo "-----> git status done"
# Use first argument as commit message, or prompt if missing/blank
if [ -n "$1" ]; then
CHANGE_MSG="$1"
else
echo -n "Enter the Description for the Change: [Minor Edit] "
read CHANGE_MSG
CHANGE_MSG=${CHANGE_MSG:-Minor Edit}
fi
# Commit and push
git commit -m "${CHANGE_MSG}" >> /config/logs/git.log 2>&1
echo "-----> git commit done"
git push origin main >> /config/logs/git.log 2>&1
echo "-----> git push done"
echo "-----> all done"
