Installing Home Assistant on a RPi 4b 8GB with SSD boot

Tags: #<Tag:0x00007fc3fb5e76c8> #<Tag:0x00007fc3fb5e75d8>

[UPDATE 2020-10-14 by @Jpsy ]:
Added instructions to update HA OS through CLI.

[UPDATE 2020-10-18 by @Jpsy ]:
More details added on installing and using Raspberry OS to update the EEPROM firmware.

[UPDATE 2020-10-20 by @Jpsy ]:
Infos added on using Raspberry OS “headless” through SSH (no monitor or keyboard attached).
Thanks to @LondonBenji for these details.

[UPDATE 2020-10-22 by @Jpsy ]:
Infos added on working (and non-working) SATA to USB3 adaptors.
Some infos about Sabrent adaptor removed (don’t use it!).
Thanks to @Noobz for these details.

[UPDATE 2020-10-31 by @Jpsy ]:
Added warning to NOT use HA OS version 5.4.
Added hint that Raspberry Pi Imager can be used to update EEPROM.

[UPDATE 2020-12-08 by @Jpsy ]:
Added warning to NOT use the snapshot restore feature of the first login screen.
All HA OS version references updated to v5.6.

[UPDATE 2021-01-08 by @Jpsy ]:
SSD boot and Pi4 with up to 8GB are now officially supported.
Updated adaptor recommendation from ELUTENG to StarTech.
Added hint to and recommendation for ODROID-N2+ and Home Assistant Blue.
Added info about a sufficient power supply.
Added info about different boot EEPROM versions.
All HA OS version references updated to v5.10.
Added warning about HA OS versions 5.6 to 5.8 with boot EEPROM update code.
Updated references to current stable boot EEPROM to 2020-12-14 2020-12-11.
Added general warning about Pi4 boot firmware problems and link to release list.
Added warning about USB3 interference with Zigbee, Zwave, …
Added section “Compilation of facts” to sum up details and infos.

[UPDATE 2021-01-11 by @Jpsy ]:
raspi-config menu item sequence updated. Thanks to @Egon for the info.
Reference added to Intel’s document on USB3 interference with WLAN, Zigbee, Zwave, Bluetooth… Thanks @tldp, @KCYeoh and all the others who mentioned it.

[UPDATE 2021-01-18 by @Jpsy ]:
Added chapter “Problems on reboot / warm boot / soft boot”

[UPDATE 2021-02-18 by @jay_p ]:
Update Firmware release status: default and latest are symbolic links to the older release names of critical and stable.


Preamble

This guide is long and has many comments. It may seem as if the process to get Home Assistant running from SSD on a Pi 4 is complicated and full of problems. But: IT IS NOT!
The actual process is fast and – for the vast majority of users – is really simple. Most of the lengthy text of the guide evolves around rather rare problems that do affect some users. But the vast majority of installations is trouble free and rock stable.
Just make sure that you obey the hardware recommendations and you will very likely be just fine.

RPi 3 for HA is dead!

I finally found that the lags and hangs of HA on my RPi 3b were the result of too many add-ons that quickly filled up the 1 GB of memory. I considered switching to a NUC or an AMD 4500U based system. But I love the low power consumption of the Pis and their small carbon footprint. So I decided to try the current upper end of the Pi spectrum:

Long live HA on Raspberry Pi 4b

…with (up to) 8GB of memory and booting directly from SSD !

UPDATE:
You may want to consider an ODROID-N2+ with 4GB instead of a new Pi. The N2+ is actually recommended by the HA devs.

Before we begin I want to make you aware of the fact, that the HA developers recommend another hardware: The ODROID-N2+. There is even a beautiful HA branded casing available for it, bundled with everything that you need to start with. See Home Assistant Blue. I don’t know if the ODROID has any of the compatibility issues in relation to USB adaptors that the Pi 4 has (and that we will cover below). But if you have not yet made your decision and ordered your hardware I honestly recommend you go with the HA Blue package. The only reasons why I stay with the Pi4 8 GB is that I have it running (with no problems) and that it has 8 GB while the ODROID has only 4 GB.

But this guide is about the Pi 4b and you are here because you looked for it. So let’s get started…

On the way from Pi3 to Pi4 and from SD to SSD I faced some problems:

  1. The 8GB version is not yet officially supported by HA.
  2. The full SSD boot of HA is also not yet officially available for the Pi 4.
  3. The SSD boot on the Pi4 is still full of pitfalls. This is not a Home Assistant problem but a Raspberry Pi problem. Especially the EEPROM bootloader firmware does not work with all available USB-to-SATA adaptors on the market.

Fortunately these problems can be solved and the resulting system works flawlessly and is extremely fast! The 8GB memory provides HA with enough space to run many add-ons and a long data history (30 days in my setup) without starting to swap or choke. And there is always plenty of in memory buffer and cache available to keep DB accesses really fast.

The following instructions will work for all version of Pi 4b – 1, 2, 4, 8 GB. You may actually have some more options with the versions below 8 GB (like using the 32bit version of HA). But I still recommend the 8 GB version as it will surely keep you happy for a longer time to come.

So let’s go in there, step by step:

Use the right SATA to USB3 adaptor

This is propably the one most important step for your success. Not all available SATA adaptors are working on the RPi 4. This is not a HA problem but a generic RPi issue. Please see this very helpful blog posting of James A. Chambers. It contains lists of working and non-working adaptors. I started myself with a Sabrent adapter based on a JMicron JMS583 bridge chip which gave me quite some trouble. I.e. it only worked reliably when connected to one of the USB2 ports – not on USB3. I then switched to an ELUTENG adaptor that works like a charm for me and was cheap too.

UPDATE 2021-01-08:
Unfortunately it turns out that there are two versions of the ELUTENG that look nearly identical but use different chip sets. One of them only works on USB2. So if you are looking for a working solution my current recommendation is that you use either of these two adaptors:

  • StarTech 2.5″ SATA to USB 3.1 Adapter
  • StarTech 2.5″ SATA to USB 3.0 Adapter

Use a sufficient power supply

The second most important factor for your success is to use a power supply that is capable of driving your Pi 4 and (!) your SSD. Nothing will give you more headaches than an insufficient power supply. You system will stall when you don’t expect it and you will not understand why!
At least use the original Pi 4 power plug with 3 Ampere. If you can get a good quality supply with 3.5 Amps or more: Use it!
Alternatively you can use a powered USB hub or casing to give your SSD a dedicated power supply. I personally don’t like that idea because a second power plug creates additional energy losses and is another item that can break. But it is still much better than an unstable supply.

Use Raspberry OS to update the EEPROM firmware

RPi 4 has a boot EEPROM (which RPi 3 did not!). You have to update that EEPROM to the latest firmware (2020-12-11 or higher) which enables USB boot. To update the EEPROM you have to temporarily install Raspberry OS. I used an SD card for this. If you want to learn more you can read the Raspberry Pi docs but we will go through all the steps here in detail.

(Hint: There seems to be a simpler way to update the EEPROM firmware through Raspberry Pi Imager instead of Raspberry Pi OS. I did not try that option. If you want to go for it, check this comment below.)

First download Raspberry OS and burn it onto an SD card. You do that using Raspberry Pi Imager. Versions for Windows, Mac and Ubuntu are available here.

A) If you have a keyboard, mouse and monitor attached to the Pi

This is my recommendation but you will need a micro-HDMI adaptor cable for that, which you might not have. If you do, simply start your Pi with that SD card and monitor/keyboard/mouse attached. Go through the initial setup process.

When all is done open a terminal window.

B) If you have NO monitor attached to the Pi:

Prepare to access Pi’s shell through SSH. To do so you simply add an empty file called “ssh” (no file extension) to the root of the boot partition on the SD card. You should have two partitions. The first (and smaller) one is the boot partition. Place the file there.

Now start your Pi with that SD card and access the Pi’s shell through SSH from another device on the network.

The ssh-file on the boot partition is deleted on first boot, so you will loose SSH access when you reboot. To keep SSH access launch raspi-config now from the command line:

sudo raspi-config

Go through…
Interfacing Options » SSH » Yes » Ok » Finish

Update the boot EEPROM firmware

This is done by telling the OS to use the latest stable version for all updates and then simply rebooting:

Edit the rpi-eeprom-update file and change the FIRMWARE_RELEASE_STATUS from “default” to “latest”:

sudo nano /etc/default/rpi-eeprom-update

Change the file, save and exit nano. Then…

sudo apt update
sudo apt full-upgrade
sudo reboot

Let the system reboot. The EEPROM should be updated automatically during the boot process.

Check the EEPROM version

sudo rpi-eeprom-update

This should now report at least firmware 2020-12-11 or higher.

Warning:
The boot EEPROM development of RPi 4 is currently a mine field!
It is possible that your USB-to-SATA adaptor contains a chipset that is not (yet) compatible with the current boot code. There are many problem reports. The StarTech adaptor mentioned above should steer you clear of that. But there is no guarantee. Some users have better luck with older (!) boot EEPROM versions. See the Pi4 bootloader EEPROM release notes for a list of stable and beta versions.

Use raspi-config to set boot config to SSD boot

sudo raspi-config

Go through these menu items:

  • Advanced Options » Bootloader Version » latest » Ok
    when the system asks to “Reset boot ROM to defaults” select No (!!!) to use the latest boot ROM.
  • Advanced Options » Boot Order » USB Boot » Ok
  • Finish » Reboot » Shut down

We are now done in Raspberry Pi OS. I recommend that you keep that SD card with RPi OS! You can always use it later for more EEPROM updates. It is also very helpful to access your HA SSD and salvage your config and data in case HA should refuse to start some day!

Flash Home Assistant to SSD

Now download the latest Home Assistant 5.x OS from here:

Look for the latest “Release 5 build X (Stable)” (5.10 at the time of this writing).
Open the “Assets” panel under the list of changes to see the downloads.
Find the image hassos_rpi4-64-5.X.img.gz - make sure it is rpi4 (not rpi3) and 64 bit.
AFAIK the 8 GB Pi won’t run with the 32 bit version.

WARNING: Do NOT use HA OS 5.4! There was an issue with that version that affected some but not all RPi4s. (See discussions below.) Also do NOT use HA OS version 5.6 to 5.8. These contained their own boot EEPROM update code that flashed a beta version into the EEPROM. This approach was removed in 5.9.

Use Balena etcher or a similar tool to flash the image onto your SSD.
(Yes Balena can flash to SSD, not only SD!)

Start Home Assistant

Plug the SSD into the Pi, take out any remaining SD card and boot.
Check http://homeassistant:8123 on your network or use Port 8123 with the IP that your router’s DHCP has assigned.

It helps to follow the boot process on a monitor that is directly attached to the Pi. You will need a micro-HDMI adapter cable for that.

When the initial login screen comes up DO NOT use the snapshot restore feature of this screen. There have been multiple reports that installing a snapshot from here does not work and may even ruin your virgin HA installation. There seems to be some incompatibility of snapshots created with other HA OS versions. Instead define your new admin user and do a fresh first start. Then go to Supervisor » Snapshots and use the dot menu in the upper right corner to upload and restore your snapshot.

Boot meditation?

When you are waiting for the initial login screen you may experience extremely long boot times (>10 min). If this happens to you you have multiple options:

  1. Check the section “Use the right SATA to USB3 adaptor” above. Chances are that you are using a non-compatible SATA to USB3 adaptor or casing. Try the StarTech that I recommended above.
  2. Try to connect your SSD through a USB 2 port (black) instead of a USB 3.0 port (blue). I know it sounds strange, but please really do! If this works for you, it may be your best option. Although USB 2 is much slower than USB 3.0 you will get a very fast system. But if option 2. works for you you should also reconsider option 1. It is very likely that your adaptor or casing is the culprit.
  3. Check if your SSD casing or your SATA to USB adapter is based on a JMicron JMS583 bridge chip. There is a firmware update for this chip that solves the problem. For Sabrent devices the update for all affected devices can be found here. But I also remind you that my own Sabrent adaptor never really worked for me. Read 1. again!
  4. Make sure your power supply is capable of feeding your Pi and (!) your SSD.
    Check the section “Use a sufficient power supply” above.

The good news after that final trouble is:
You are done! Enjoy your new HA installation.

Problems on reboot / warm boot / soft boot

You are done? Well, maybe not completely. Sorry.

Some users experience problems when they restart their Pi without powering off. There is a lot of discussion on the Pi issue tracker about this problem and it is not completely solved yet (also not for RPi OS). It seems to be connected to the fact that the Pi cuts USB power on reboot for a short moment. For some drives this power-off time seems to be ok, for others too short or too long. To make things worse, RPi4 8 GB seems to behave differently from Pis with less RAM: 8 GB does a longer power-off on reboot (which seems to create less over-all problems). If HA hangs on reboot, but boots fine when you cut the power and plug it back in, then you are affected. There are options in EEPROM config that may solve this problem (USB_MSD_PWR_OFF_TIME, POWER_OFF_ON_HALT). See SOLVED: SSD Enclosures with Jmicron chips JMS580 and JMS583 don't boot up. RPi4. · Issue #266 · raspberrypi/rpi-eeprom · GitHub for details.

A warning about USB3

USB3 signal frequencies may interfere heavily with WLAN, Zigbee, ZWave, Bluetooth, enOcean!
Use USB extension cables (male-to-female) to place your radio sticks away from the Pi.
Connect your USB2 radio sticks to USB2 ports not to USB3. (Well, actually the USB3 ports should not really interfere as long as they run in USB2 mode, but better don’t push your luck).
Read Intel’s document on USB3 interference to understand the details and how you can place and shield your devices.

Update the DEV version of Home Assistant OS

Nerd info on HA OS. This was mainly insteresting for early adopters while the 5.x OS was still under development:

If a new 5.x DEV version of HA OS becomes available you may find that the Supervisor menu in the HA frontend does not offer you the update, even if you have switched to the beta channel. This is due to the fact that only stable and beta versions are offered here, not dev versions. But you can update your OS at any time using the command line interface (CLI) of HA. To do so you need SSH shell access to HA. There are multiple add-ons that expose SSH (my recommendation: Frenck’s great “SSH & Web Terminal”).
In the shell you can use the following command to update your OS to a defined version:

ha os update --version 5.10

Try this command before and after the update to get some system info:

ha os info

Compilation of facts

This is a compressed collection of facts that are spread throughout the guide above.
Might be helpful for trouble shooting.

  • HA OS versions 5.0 to 5.5 did NOT change the boot EEPROM
  • HA OS versions 5.6 to 5.8 DO change the boot EEPROM to version 2020-10-28 - BETA. This was beta but considered the better solution for many users because it solves reboot hangs for many drives.
  • HA OS versions since 5.9 do NOT change the boot EEPROM any more to give back control to the user.
  • There are multiple generations of “stable” boot EEPROM versions available (see release list).
    But even in the RPi GitHub repo there are plenty of issue reports. So I suggest that if you found an EEPROM version that is working for you, you do NOT upgrade the EEPROM at this time.
  • All in all it seems recommendable right now to manually try several stable boot EEPROM versions and avoid HA OS versions 5.6 to 5.8 because they will overwrite the EEPROM on their own. Start with the latest stable EEPROM version as described in the guide.
  • There seem to be problems with some adapter chip sets when switching to UAS (USB Attached SCSI, UAS or UASP) during Linux system startup. But I still have to gather information about that.
  • Some drives seem to hang on reboot. This behavior differs between RPi4 with 8 GB and the versions with less RAM. All do a USB power off but the 8 GB version does it much longer. If your Pi with up to 4GB hangs on reboot try to really power off/on the system. There are options in EEPROM config that may solve this problem (USB_MSD_PWR_OFF_TIME, POWER_OFF_ON_HALT). See SOLVED: SSD Enclosures with Jmicron chips JMS580 and JMS583 don't boot up. RPi4. · Issue #266 · raspberrypi/rpi-eeprom · GitHub.
  • Make sure your RPi power supply is sufficient to power both your Pi AND your SSD. Better use the original RPi4 power supply with 3 Ampere output. If you find a quality supply with more than 3 Amps: use it.
  • USB3 signal frequencies may interfere heavily with WLAN, Zigbee, ZWave, Bluetooth, enOcean. Use a male-female USB adapter cable to get your radio sticks away from the Pi. Connect your sticks to USB2 not 3.
91 Likes

Thanks for this detailed article. By follow it I was able to put together
a RPI 4B 4Gb, a Samsung 860 EVO 260Gb SATA memory block, and an Eluteng
SATA-to-USBb 3 adapter, with the 64-bit version of HA 5.2 flashed on.
After about 10 minutes the normal opening screen cam up, and a current
snapshot from the running system was read in. To my pleasant surprise,
the system came up with everything in place and running properly.

Again, thanks so much for the effort put into researching and publishing this
blog/article.

CTFrank

BTW: an initial attempt using a WD Blue 500 SN500 NVMe SSD with a Plugable
SSD-to-USB 3 housing/adapter failed to boot; haven’t yet figured out why…

What are the chances! I just decided about an hour ago that I wanted to tinker with HA. Found your post. Took about 15 minutes to get my RPI 4B 4Gb up and running (Adata SU635 240Gb with Startech USB adapter). Thanks for making this so easy! So nice to see HA picking up on all my devices. :slight_smile:

1 Like

First post, first time ever installing and using HA and using and your guide got me there! Thank you @Jpsy !

Now to start playing around with HA and working things out!

1 Like

Why did I never consider if my Sabrent SSD adapters could be firmware upgraded :tired_face:
Just updated 3 of them, so thanks for that.

1 Like

@Jpsy what combination of SSD and eSata to USB adapter are you using.

@ConcordGE I am using a 120 GB Crucial BX500 connected to a Sabrent EC-SSHD adapter. Not sure wether this is the best solution, as I still have reduced performance when I connect this combination to the USB 3 port of the Pi. I really flies on USB 2, but maybe with another adapter you might be fast on USB 3 too (or even faster).

I would be very interested to hear from other users with other equipment whether they also experience longer boot times and slower response (i.e. when opening the history) on USB 3 compared to USB 2.

1 Like

Hi Jpsy
I did a test while troubleshooting my JMicron adapter, I did a read/write test on the usb 2 and 3. I had about 1/2 the read/write speed on the usb2.
Real world difference my wary.
This link may help: https://jamesachambers.com/raspberry-pi-4-usb-boot-config-guide-for-ssd-flash-drives/

1 Like

Hi @Noobz, very interesting indeed!
I have immediately ordered the ELUTENG adapter after reading that guide. It is even cheaper than the Sabrent and has good reviews. Will report back here as soon as I have tested it with HA.

Unless I could achieve a notable performance increase on an SSD via USB3 on the Raspberry Pi 4 I wouldn’t be interested going that route. The bottleneck via USB2 would be unacceptable.

I just can’t see enough examples of the hardware combination required to achieve this.

But what is your alternative, @ConcordGE? An SD card? The SSD on USB 2 still outperforms any SD card by a magnitude. And SD cards can never be a permanent solution for HA because they wear quickly under the steady stream of database writes. I killed 3 of them with my HA installation before I came to recognize that. :smiley:

2 Likes

I’ll hold back till a credible usb 3 solution appears.

Not wanting to get into the failed SD card debate but I’ve only experienced that once and unfortunately the card in question was from a dubious source. So many fakes doing the rounds these days.

Interesting piece on the subject here. Worth a read. https://www.raspberrypi.org/forums/viewtopic.php?t=199414

1 Like

Do youv know how to check chipset of USB to SSD adapter? I mean using windows or linux?

Also sellers (even manufacturers) often don’t include this information on their webpages. Is there some database of adapters with needed information?

thank you in advance

If you use linux, all you need to do is type lsusb with and without the usb adapter connected.
I dont know how to do it in windows.

I am on absolute newbie here.
I am on hold here

### Raspberry Pi Imager

The easiest way to to update the bootloader to the latest version with default settings is to use the [Raspberry Pi Imager](https://www.raspberrypi.org/downloads/) to install a boot recovery image onto a spare SD card.

Select "Choose OS -> Misc utility images -> Raspberry Pi 4 EEPROM boot recovery"

Raspberry Pi OS also keeps the EEPROM up-to-date: new EEPROM images are applied when the Raspberry Pi 4 next boots.

on this , I have flashed the EEPROM onto SD card. But I couldn’t find a way to remote ssh from my laptop.

can someone help me on this pls? I do not have a monitor/Keyboard.

1 Like

Sorry, but for that step a monitor and keyboard connected directly to the Pi are needed.

There seems to be an option in Raspberry OS to enable SSH by adding an empty file named ssh to the Pi’s boot partition. With that step it could be possible to execute all needed steps from the command line through SSH. But I did not test that option.

This is exactly what I did and how I set it up @uniquecool , no keyboard/mouse/display connected to my new RPi4. Just place an empty file called ‘ssh’ no file extension, nothing, just put that file on the boot microSD and the Pi checks for it on boot and if it finds it, the SSH server is automatically started and then the file is deleted, so make sure you go into raspi-config to enable SSH still otherwise if you reboot it for whatever reason you won’t have SSH access anymore.

2 Likes

Hi @LondonBenji, thanks for the confirmation. I have added your info to the guide!

1 Like

@Noobz I received the ELUTENG adapter today.
It boots HA from my SSD on USB 3 within 1.5 minutes!
GREAT!!! Thanks for that info.
I will update the guide accordingly tomorrow.

2 Likes

@Jpsy
I’m glad I could be of help.
Waiting for my own shipment ELUTENG adapter.

1 Like