Custom firmware ESPHome-Xiaomi_bslamp2

HI, fisrt of all thank you for the great work.
I compiled and installed your firmware but the lamp refuses to boot throwing the following error:
[W][esp-idf:000]: E (361) system_api: Base MAC address from BLK0 of EFUSE CRC error, efuse_crc = 0x27; calc_crc = 0x13
From you yaml i understand that there is a special package for disabling that check. any idea why that doesnt work for me?

You are now exactly at the starting point of my development work! :slight_smile:
Based on the error, you are not actually using the fixed platform package. That must indeed be configured in the yaml config. Can you please copy and paste the esphome: section of your firmware here, so we can take a look at it?

Sorry, my fault. :smile:
The docs pointed me to the right direction

The latter branch’s name gives a clue as to the major difference from the former: it has MAC address CRC validation essentially disabled (it still checks it, but the log message is downgraded from error , and the abort() call is removed)

It’s just a warning and the lamp boots successfully. I expected a litte bit more logging output but it makes sense that there isnt any while having set the log level to WARN.
I switched to DEBUG and tadaa… i can see that the lamp boots and connects to wifi.
I am able to connect to homeassistant via api but isnt there any webinterface on the lamp itself?
I am asking because i had some compile warnings and no webinterface

No, there’s no webinterface on the device.
I just tried a clean build, and the only warnings I saw were about esp32-hal-spi.c.
What are the compile warnings that you see?

Library Manager: Warning! Could not install dependency {'name': 'Hash', 'platforms': ['espressif8266']} for package 'ESPAsyncWebServer-esphome'
Library Manager: Installing Hash
Warning! Could not find the package with 'Hash' requirements for your system 'linux_x86_64'
Compiling .pioenvs/bedside_lamp/FrameworkArduino/esp32-hal-time.c.o
/home/esphome/.platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-spi.c: In function 'spiTransferBytesNL':
/home/esphome/.platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-spi.c:922:39: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
                 uint8_t * last_out8 = &result[c_longs-1];
                                       ^
/home/esphome/.platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-spi.c:923:40: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
                 uint8_t * last_data8 = &last_data;
                                        ^

Ah, I have seen that Hash warning too, but I haven’t really looked into it. I stopped reading when I saw “espressif8266” (since the lamp uses an “espressif32” instead), so I hadn’t noticed the ESPAsyncWebServer-esphome reference there.

I do find some code in the development branches of ESPHome, in which warnings about Hash are being actively filtered from the platformio output. When I build my firmware on such development branch, the message is no longer visible during the build.
Base on that, I conclude that it is safe to ignore these Hash warnings.

Update: I just released v1.0.0-RC1

After extensive testing, I think that it is about time to bring out a release candidate.

There have been various code improvements since the last release, but the main change is the reorganization of the project documentation. A whole new section “Configuration guide” has been added, in which you can find information about all the configuration options.

The main reason for not making this a 1.0.0 stable release right now, is that I would love to make use of a wonderful new feature in ESPHome that is being worked on at this moment: the possibility to point ESPHome at my GitHub repo from the YAML configuration, to have it included automatically in a build. This takes away the need to download the code and move it into a custom_components directory manually. This vastly improves the process for people who want to make use of the xiaomi_bslamp2 component implementation.
As soon as ESPHome includes this feature in the stable release, I will update my repository structure for supporting this and bring out 1.0.0 stable. Progress on this can be followed in this issue on the repo.

I hope you will enjoy this release.
I know I do! :grinning_face_with_smiling_eyes:

4 Likes

Good to see even more progress - Thanks for the hard work!
A further step could be to collect the work on converting Xiaomi/ YeeLight devices into one place on the ESPHome Components pages.
Currently there two conversions that I know of: This one and the smaller desk lamp. I am hoping that some of the other lamps can be converted. The desk lamp conversion is not ESPHome but the knowledge could be reused to make an ESPHome version.
Xiaomi Desk Lamp

I think it’s a good idea finding a better structure for further experiments on other Yeelight/Xiaomi lamps. I’ve seen bslamp1 and D2 mentioned further up in this thread but this thread was started mainly for the bslamp2, and has evolved to mmakaay’s “playground of black sorcery” around the bslamp2 especially. But as you say, there is a good chance that mmakaay’s work can be reused for several other lamps since they may seem to be structurally similar! :slightly_smiling_face:

2 Likes

As I envision it, my set of components for the xiaomi_bslamp2 could be be described in the cookbook section of the ESPHome documentation. Would that be the kind of “one place” that you’re talking about?

I don’t think that it would be a good idea to move the full codebase into the ESPHome codebase. This would result in individual component descriptions amongs many component categories (since the code provides a light output, binary sensors, a sensor, and a text_sensor to cover all of the lamp’s features).

Distributing component code always was a bit of a hassle. That hassle would be a good reason for trying to get my code into the ESPHome repo. However, with the new “external components” feature on its way, it will become really, really easy to use the code from my repo.

Therefore, the cookbook idea seems like a perfect middle ground: expose and explain the conversion of the lamp in the cookbook, but keep the code contained in its own repository.
There’s of course one exception to this: if the ESPHome maintainers would request me to incorporate my code base, I would happily do so.

The Bedside Lamp 2 has some really idiosyncratic elements to it, especially the I2C-controlled front panel, night light mode, the brain-wrecking way of driving the LEDs and the fully customized way of having to handle transitions. So that raises the question how much of the code base would be reusable for other (Xiaomi) lamps.

Having said that, it would be good to have a similar setup for other lamps. I think that one of the useful reusable concepts that I introduced in this code, is that I moved the complexity of having to define the correct pinouts into the codebase itself. The reasoning behind that is the fact that this is a manufactured lamp, with a standardized board, for which all the pins are known.

By doing this, the configuration for the end user only focuses on exposing the lamp’s functionality, not on its exact wiring scheme. This makes it easy to use, and the user won’t be able to make any mistakes. If one would for example set the PWM frequency of the blue output to 30000 Hz instead of 3000 Hz, the result would be a lamp that wouldn’t work as expected. There’s no risk in that with my code, since those settings are handled within the code, not within the YAML.

Extrapolating this to other lamps / devices: even if you have a lamp that can be controlled by using only standard ESPHome features, by configuring everything in the YAML file, it might be worth to create a component for it, that encapsulates the pinouts and possibly other static technical data, to make the end user config simpler.

playground of black sorcery

Whoops, and I was thinking that I hid that well enough :laughing:

Cookbook sounds like a good place.
I need to understand the standard ESPHome features that would map onto a simple Yeelight/ Xiaomi lamp (White + colour temperature + rotary encoder).

Prior to the LAN lockout, I was thinking about buying one of these - Question is: Do I buy it and risk not being able to hack it…

Seems that the more recent Xiaomi lights have an ESP32 single core SOC. I have seen a workaround for the smaller Xiaomi desk lamp with this processor and ESPHome.

I don’t own any other Xiaomi lamps, but the desk lamp pro has caught my attention before. If I get one, I will for sure hack it. If only to get rid of the lamp’s behaviour of phoning home to the mi cloud.

The ESPHome concepts would likely be the rotary encoder component and a custom light output. There might be the need for assigning a pin as a power supply pin for the light output. I don’t think there will be a lot more to it.

LTE tracking

I started working on the next fun feature for the lamp. It would be cool to get BLE tracking working, so the lamp can be used for presence detection.

@davorin and I have already been playing with it, as it should work out of the box. However, when enabling the esp32_ble_tracking component, the device is continuously rebooting. The reason for this is that the WDT (watch dog timer) is really unhappy with the loop() of that component. Possibly this is because we’re working with a single core chip here. Not sure, just a hypothesis.

The work on this can be followed in issue #18 on GitHub: “[FEATURE] Enable BLE presence detection”.

More work on API client disconnects

Unfortunately, the fix that I wrote for some race conditions in the AsyncTCP library, gave extra disconnects for at least one user of a different device. That is not okay, so I have been looking into fixing the disconnects in another way. This worked out very well. I’ve got an even more stable client connection now than with my former fix :smiley:

It does need a very simple and safe addition to the api_server component. I already created the pull request for this. All details for the fix can be found in this pull request.

1 Like

Is it possible to flash the Bedside lamp using arduino? I’ve tried to read_flash with esptool, but i get:

esptool -p /dev/ttyACM0 read_flash 0x0 0x400000 original-firmware.bin
esptool.py v2.8
Serial port /dev/ttyACM0
Connecting........_____....._____....._____....._____....._____....._____....._____

A fatal error occurred: Failed to connect to Espressif device: Timed out waiting for packet header

Did you connect gpio0 to ground, before powering up the lamp? That must be done to get the lamp into flashing mode.
Another thing to check would be the correct wiring of TX and RX, which must be connected to respectively RX and TX. If the connections aren’t crossed over like this, you’d see the same symptom.

What way would you flash using Arduino? By connecting to the serial pins of the Arduino? That sounds like an option, but I never did it that way. I also highly doubt if that would yield different results.

Also, do not power the circuit from your TTL-adapter, using only the ordinary power supply, as before mentioned. Only connect TX<->RX, RX<->TX and GND, as well as pulling the GPIO0 to GND when powering up the lamp. (Do NOT connect 3,3v in any way at the same time since it will surely fry something) :wink:

Update on getting BLE tracking and WiFi to work together

I’ve worked on various work-arounds for getting BLE + WiFi operational. Unfortunately, my conclusion is that doing BLE tracking currently is not an option. When using BLE, the WiFi stack is broken for OTA. Disabling the Bluetooth doesn’t help here. Future releases of esp-idf (v4 branch), might bring some goodness in this respect, along with making use of the Nimble component. This is likely not a short term solution, because it requires changes in multiple stacks, and we’ll have to wait for a stable v4 release of esp-idf.

For now, I’m closing the related issue on GitHub, and will work on something else.

Created my own arduino-esp32 framework package build

In the process of persuing a working combination of WiFi and Bluetooth LE (a persuit that has failed so far unfortunately), I ended up building arduino-esp32 framework packages for recent development versions of arduino-esp32 and esp-idf (v4 alpha). One result of this is that I now have my own build scripts for creating the framework package.

I have shared the work on this on my github. As a result, the following is now also a working esphome configuration for the Bedside Lamp 2:

esphome:
  name: ${name}
  platform: ESP32
  board: esp32doit-devkit-v1
  platformio_options:
    platform: [email protected]
    platform_packages: |-
        framework-arduinoespressif32 @ https://github.com/mmakaay/arduino-esp32-unicore-no-mac-crc#v1.0.6

You can omit #v1.0.6, to make use of a recent development build. This could be useful if there’s some fix in there that was done after the v1.0.6 release of arduino-esp32. This build is not automated, but if you need a more recent build, then simply drop me a line.

The scripts that I wrote to build the releases, can be found at:

About esp-idf v4

My repositories don’t yet contain a build using esp-idf v4, because this would not work out of the box. Some changes are needed in the esphome code to make this work. I had to do some modifications to the WiFi code, but there might be other components that would need updates as well. Therefore, it is of little use to already publish such version.

The interesting thing for me about v4, is that there has been some work on coexisting use of WiFi and Bluetooth. That was the reason for me to look into the current v4 alpha release. With v3, there doesn’t seem to be a way to get things working. Sadly, WiFi + Bluetoon = broken OTA upgrades is my experience up to now.

Another interesting thing about the v4 release, is that there will be support for esp32-s2 and esp32-c3. Those are new targets, next to the esp32 target, which is the only available target in the currently stable v3 release of esp-idf.

Should the lamp’s light turn on after powering it up to boot? I’m sure that TX and RX are connected fine. gpio0 is also connected to the ground and I still get the error and the esphomeflasher is not showing any logs :< This is my wiring:

Can’t really say using arduino, but have you tried setting the baud rate when read_flash?

I have never used an Arduino for flashing like this. How does it work? You run a sketch that bascially turns the Arduino in a USB to serial adapter like we use?
Do you have an URL to the documentation for this, so we can take a look at it?
→ Update: I found some information on this setup. Apparently running an empty sketch is all that is required here. Pulling RESET down to GND also was mentioned as a way to make the UNO act as a serial to USB converter. Nice!

Have you tried turning around the RX and TX connections BTW? It’s a bit hard to see how TX and RX are connected to the board, because of the tape. I do think that they are correct. The green one seems to go to the pad that is next to the GND pad. That is the TX. It goes to RX on the Arduino UNO, so it should be okay.

And I agree with @Smurf on making sure that the baudrate is correct. I always use 115200 for flashing purposes.

One warning It’s been a while since I’ve touched Arduino UNO, but I just realized that its outputs are using 5V, whereas the ESP32 uses 3.3V for signalling. I don’t think the ESP32 is 5V tolerant. I wouldn’t make a direct connection between 3.3V and 5V hardware that way.