Custom firmware ESPHome-Xiaomi_bslamp2

AndyHiggins · September 29, 2024, 12:57pm

Thank you to @mmakaay for the amazing work on this project. It was very satisfying to get the lamp to work with ESPHome. In case this is useful to others, I am sharing my experience in getting it to work based on the specific tools I used and it also seems some things have changed since the original instructions were written.

The instructions for connecting the serial adaptor to the lamp are great. Being the first time I have done this, I found it easiest to put a small blob of solder on the end of the exposed wire, hold that end of the wire over the connection point on the lamp and touch the solder blob on the wire with the soldering iron (thereby not requiring a third arm).
I could not use the ESPHome Flasher software as my Mac gave a security error when trying to run it. I also could not use the ESPHome browser in directly in HA as I do not have a local HTTPS connection (this does not seem that simple to set up). I therefore downloaded the ESPHome firmware and uploaded it using ESPHome Web directly from my computer over the serial adaptor.
After installing the ESPHome Add On in Home Assistant, create a new ESPHome device, give it a name, next, select ESP32, skip, download the YAML config file for this new device (so you can refer to it). Edit the YAML file in HA replacing the contents with the example.yaml changing some of the values based on the initial device YAML file downloaded e.g. api (you now have to use the encryption option instead of a password), ota, add the ap option and I also added the webserver option. Save.
Click on the 3 dots next to the ESPHome device in HA and select “Install”. Select the Manual download option. Once compiled (this this can take a while the first time), select the factory format to download the firmware.
Add external power to the lamp with the GPIO0 pin connected to ground (to put it in flashing mode). Connect to the lamp from your computer over the serial adaptor and upload the firmware.factory.bin downloaded above.
Remove external power from the lamp, disconnect the GPIO0 pin from ground (to prevent the ESP32 from going back into flashing mode) and add back external power to the lamp. Check the log files on the ESPHome Web interface.

Some additional notes:

I initially followed the instructions provided on the ESPHome devices website which for example references a main branch example.yaml from September 2021. Be sure to use the latest example.yaml from the dev branch.
The serial adaptor I used: USB to Serial Adapter CH340G - 5V/3.3V. With the supplied jumper you can configure the module for 5V or 3.3V. Without the jumper, the module is automatically powered with 3.3V. As we want 3.3V, I removed the yellow jumper.
As I needed to connect both GND and GPIO0 on the lamp to the ground on the serial adaptor, I used a breadboard to connect the 3 wires together.

mmakaay · September 30, 2024, 11:51am

Nice write-up and a cool picture of the setup @AndyHiggins !! Always fun to see in how many ways the PCB can be hooked up for flashing

Good point about the encryption key, which is now mandatory. Based on a problem report, I found that the fast random light effect can result in a lamp reset after a while (which worked flawlessly in earlier releases of ESPHome with the same firmware, so suspecting some memory leak or so in the more recent ESPHome code). when I start working on cracking that issue, I will also give the configuration files some love and attention to be up-to-date with ESPHome.

Enjoy the lamp!

bouwew · October 20, 2024, 8:39am

Hi all,

I like to keep the ESPHome software updated on the 2 lamps I have.
I just wanted to report that ESPHome v2024.9.0 and higher seems to break the communication with HA. I’m not sure which 2024.9.x version, I didn’t try them all.

I had recently updated to v2024.10.0 and tried to update one of the lamps. Updating goes well but then there is a message that the HA API could not be connected.
I tried to roll back to earlier versions of ESPHome, none worked until I rolled back to v2024.8.3.

mmakaay · October 20, 2024, 10:26am

Yes, there are some recent changes in ESPHome that turn out to be breaking changes for the type of ESP32 that is used in these devices. There is a pull request on GitHub that should fix the issue:

github.com/mmakaay/esphome-xiaomi_bslamp2

Adding ignore_efuse_custom_mac config setting

mmakaay:dev ← ezcGman:esphome-efuse-changes

opened 08:10AM - 17 Oct 24 UTC

ezcGman

+1 -0

Hey there, related to this issue here, I've created this pull request: https:…//github.com/mmakaay/esphome-xiaomi_bslamp2/issues/130 However, I can't share the issue that the config won't compile at all or throws any error. For me the devices simply had a weird new MAC address. There had been some changes in ESPHome (which should **clearly** have been marked as breaking change... Or at least a bit more prominent...) to how it makes use of burned in efuse MAC addresses, especially this one: https://github.com/esphome/esphome/pull/7498 Apparently, the burned in efuse MAC addresses are useless. After updating to ESPHome 2024.10.0, my two bedside lamps showed with these MAC addresses: B4-C3-FB-3F-00-00 & AC-C3-FB-3F-00-00. It's enough to simply add the news config setting and set it to `false`, like in this PR. Greetings, Andy!

I’m currently on a short trip, but am heading home today. I will look at this PR and plan to provide a new version of the firware config packages that is compatible with the latest ESPHome this week.

Possible fix:

(pretty sure it works, but saying “possible” since I did not check it out myself yet)

As reported in the PR, you can try to tweak the esp32: config block. In your device yaml add this section:

esp32:
  board: esp32doit-devkit-v1
  framework:
    type: esp-idf
    sdkconfig_options:
      CONFIG_FREERTOS_UNICORE: y
    advanced:
      ignore_efuse_custom_mac: true
      ignore_efuse_mac_crc: true

or possibly only this bit to add the new required option:

esp32:
  framework:
    advanced:
      ignore_efuse_custom_mac: true

That last one should merge its config with the config from the core.yaml package, giving you the same end result.

bouwew · October 20, 2024, 10:46am

Thanks for providing more background! I’ll wait for your updating and testing

mmakaay · October 22, 2024, 4:00pm

Note: This release requires ESPHome 2024.10.0 or newer.

Important Update for ESPHome 2024.10.0 Users

ESPHome 2024.10.0 introduced what turned out to be a breaking change for the type of ESP32 chip that is used in our beloved lamps. See the ESPHome changelog for more information. This release provides the required changes to make the firmware project compatible with ESPHome 2024.10.0.

You already flashed your lamp and it ended up bricked? Then no worries!

If you attempted to upgrade your lamp using ESPHome 2024.10.0 before this release, you may have encountered an issue where the lamp became unavailable in Home Assistant. Fortunately, in all reported cases, it was still possible to restore functionality by flashing a corrected firmware to the bslamp2.

If you’re managing multiple lamps and experience issues with flashing, unplug all of them, then reconnect and flash each lamp individually, one at a time.

Thanks for the help spotting and fixing the issue to Andy, @Jos, @randybb and @kbx81!

Friendly name included in the esphome: config section

The friendly_name value as defined in the substitution: section of the example.yaml is now used for defining the friendly name in the esphome: configuration section. With this change, the friendly name is also used by Home Assistant for referencing the device and its entities.

Excerpt from the new core.yaml configuration package:

esphome:
  name: ${name}
  friendly_name: ${friendly_name} <-- this definition is new

Thanks to fmauNeko on GitHub for suggesting actually defining the friendly name in here.

Having the friendly name defined now, might have impact on the entity IDs as you see them in Home Assistant.

In the core.yaml, the ${friendly_name} has been removed from the default values for light_name and light_mode_text_sensor_name. The reason for this, is that ESPHome automatically uses the defined friendly name as a prefix in the entity IDs. When keeping the ${friendly_name} in there, this would result in having repeated friendly names (e.g. light.bedside_lamp_bedside_lamp_rgwbb_light).

If you have overridden the above names in your custom device YAML config, then remove ${friendly_name} from them.
You could also get back to the original behavior by adding the following to your device YAML file:

substitutions:
  name: ...
  friendly_name: ...
  light_name: ${friendly_name} RGBWW Light
  light_mode_text_sensor_name: ${friendly_name} Light Mode
  
esphome:
  friendly_name: ""

How to upgrade your device

The basic steps for upgrading are:

Upgrade ESPHome to 2024.10.0 or later;
Update your device configuration (see below);
Compile and upload the new firmware.

If you have used the example.yaml to create your configuration, then update the configuration package ref: to point to release/2024.10.0.

packages:
  bslamp2:
    url: https://github.com/mmakaay/esphome-xiaomi_bslamp2
    ref: release/2024.10.0 <--- update this one
    files:
      ...
      ...

Full changelog

There were some other changes, without end-user impact. For a complete overview of all changes, you can take a look at
the CHANGELOG.md