ESPHome GUI component using LVGL

lukasz-tuz · June 30, 2023, 6:41am

Hi!

I started working on GUI component for ESPHome some time ago and I finally got it to state in which it can be shared Project is hosted on GitHub: https://github.com/lukasz-tuz/esphome-gui.

This component uses a slightly modified ESPHome’s Display component to enable LVGL as library that renders GUI widgets - instead of built-in rendering engine. No 3rd party SPI drivers are needed, everything should work on ESPHome’s own display components.

GUI is defined and configured in YAML. Here’s an example of simple clock built on LiliyGO T-Embed board:

...
gui:
  id: mygui
  display_id: disp
  items:
    - type: label
      id: mylabel
      position: 40, 100
      dimensions: 100x25
time:
  - platform: homeassistant
    id: home_time
    on_time:
      - seconds: 0
        minutes: /1
        then:
          - lambda: |
              id(mylabel).strftime("%H:%M", id(home_time).now());

Currently, only Label and Checkbox are implemented, but I’m hoping to get more widgets in there in the near future.

DerPicknicker · July 1, 2023, 8:51am

Hi,

will it work with all displays that are supported by lvgl or need esphome also support the driver ic?

lukasz-tuz · July 1, 2023, 8:40pm

Lvgl itself doesn’t implement communication with a display or its controller, it always needs a driver. Here, all of the drivers available in ESPHome should work. I’ve only used it with st77889v so far, though.

It should be easy enough to integrate it with TFT_eSPI library, like esphome-lvgl does.

swante · July 2, 2023, 10:58am

I tried to use your component.

But I run in the following error:

esphome: None
  name: ha-touchscreen02
  friendly_name: ha-touchscreen02
  on_boot: 
    then: 
      - script.execute: backlight_script
  includes: 
    
    Could not find file '/config/esphome/esphome/components/gui/lv_conf.h'. Please make sure it exists (full path: /config/esphome/esphome/components/gui/lv_conf.h).
    - esphome/components/gui/lv_conf.h
  build_path: .esphome/build/ha-touchscreen02

YAML:

external_components:
  - source:
      type: git
      url: https://github.com/lukasz-tuz/esphome-gui
      ref: main
    components: [ display, gui ]


esphome:
  name: ha-touchscreen02
  friendly_name: ha-touchscreen02
  on_boot:
    then:
      - script.execute: backlight_script
  includes:
    - esphome/components/gui/lv_conf.h

lukasz-tuz · July 2, 2023, 3:56pm

When you list files under /config/esphome/esphome/components/gui/, is the lv_conf.h there?

swante · July 2, 2023, 4:23pm

Chicken/Egg Problem (i think you can reproduce the problem, if you clean the builds and delete the already downloaded component):

Step 0:
No lv_conf.h exists at all. (component was not downloaded yet)
Compiling with the include statement does not work.
compile it without an include statement, so that the component is downloaded.

Step 2:
Now that the file is downloaded:

 includes:
    - /config/esphome/.esphome/external_components/d99de832/esphome/components/gui/lv_conf.h

now I get the following error message:

In file included from /data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/src/misc/lv_log.h:16,
                 from /data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/lvgl.h:24,
                 from src/esphome/components/gui/gui_objects.h:4,
                 from src/esphome.h:22,
                 from src/esphome/components/gui/gui.h:3,
                 from src/esphome/components/gui/gui.cpp:1:
/data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/src/misc/../lv_conf_internal.h:35:42: fatal error: ../../../src/lv_conf.h: No such file or directory
         #include __LV_TO_STR(LV_CONF_PATH)
                                          ^
compilation terminated.
*** [/data/ha-touchscreen02/.pioenvs/ha-touchscreen02/src/esphome/components/gui/gui.cpp.o] Error 1
In file included from /data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/src/misc/lv_log.h:16,
                 from /data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/lvgl.h:24,
                 from src/esphome/components/gui/gui.h:4,
                 from src/esphome.h:21,
                 from src/esphome/components/gui/gui_objects.h:3,
                 from src/esphome/components/gui/gui_objects.cpp:1:
/data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/src/misc/../lv_conf_internal.h:35:42: fatal error: ../../../src/lv_conf.h: No such file or directory
         #include __LV_TO_STR(LV_CONF_PATH)
                                          ^
compilation terminated.
*** [/data/ha-touchscreen02/.pioenvs/ha-touchscreen02/src/esphome/components/gui/gui_objects.cpp.o] Error 1

lukasz-tuz · July 2, 2023, 7:08pm

Dang, it did not occur to me that it will happen when downloading an external component

For the time being, Please clone the repo manually to a directory next to your project dir and include the conf file like this:

esphome:
  name: esphome-lvgl
  includes:
    - ../esphome-gui/esphome/components/gui/lv_conf.h

I’ll try to remove this include dependency from the YAML file entirely.

swante · July 2, 2023, 7:43pm

No luck.

Changed yaml-file:

external_components:
  - source:
     type: local
     path: ../esphome-gui/esphome/components
    


esphome:
  name: ha-touchscreen02
  friendly_name: ha-touchscreen02
  on_boot:
    then:
      - script.execute: backlight_script
  includes:
    - ../esphome-gui/esphome/components/gui/lv_conf.h

but the compile error is the same.

lukasz-tuz · July 3, 2023, 8:05pm

OK, should now be fixed on main. Using includes: in YAML to add lv_conf.his no longer needed, component now handles its dependencies by itself.

DerPicknicker · July 4, 2023, 6:18pm

@lukasz-tuz … What would your the best approach to get this Display working?
=> 3.95寸86盒_深圳市启明智显科技有限公司

It uses an GC9503V as Controller. Unfortunately it’s not supported by the tft library

swante · July 4, 2023, 7:05pm

somehow I am not able to get it compiled.

Although I think I have deleted all the old downloads within HA, I still get the following error message.

Compiling /data/ha-touchscreen02/.pioenvs/ha-touchscreen02/src/esphome/components/gui/gui_objects.cpp.o
Compiling /data/ha-touchscreen02/.pioenvs/ha-touchscreen02/src/esphome/components/ili9xxx/ili9xxx_display.cpp.o
In file included from /data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/src/misc/lv_log.h:16,
                 from /data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/lvgl.h:24,
                 from src/esphome/components/gui/gui_objects.h:4,
                 from src/esphome.h:22,
                 from src/esphome/components/gui/gui.h:3,
                 from src/esphome/components/gui/gui.cpp:1:
/data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/src/misc/../lv_conf_internal.h:35:42: fatal error: ../../../src/lv_conf.h: No such file or directory
         #include __LV_TO_STR(LV_CONF_PATH)
                                          ^
compilation terminated.
*** [/data/ha-touchscreen02/.pioenvs/ha-touchscreen02/src/esphome/components/gui/gui.cpp.o] Error 1
In file included from /data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/src/misc/lv_log.h:16,
                 from /data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/lvgl.h:24,
                 from src/esphome/components/gui/gui.h:4,
                 from src/esphome.h:21,
                 from src/esphome/components/gui/gui_objects.h:3,
                 from src/esphome/components/gui/gui_objects.cpp:1:
/data/ha-touchscreen02/.piolibdeps/ha-touchscreen02/lvgl/src/misc/../lv_conf_internal.h:35:42: fatal error: ../../../src/lv_conf.h: No such file or directory
         #include __LV_TO_STR(LV_CONF_PATH)
                                          ^
compilation terminated.
*** [/data/ha-touchscreen02/.pioenvs/ha-touchscreen02/src/esphome/components/gui/gui_objects.cpp.o] Error 1

I have also tried it outside HA: esphome compile ha-touchscreen02.yaml
here I get the following error message:

Compiling .pioenvs/ha-touchscreen02/FrameworkArduino/wiring_shift.c.o
Archiving .pioenvs/ha-touchscreen02/libFrameworkArduino.a
Indexing .pioenvs/ha-touchscreen02/libFrameworkArduino.a
Linking .pioenvs/ha-touchscreen02/firmware.elf
/home/sven/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pioenvs/ha-touchscreen02/src/esphome/components/ili9xxx/ili9xxx_display.cpp.o:(.rodata._ZTVN7esphome7ili9xxx13ILI9XXXST7796E[vtable for esphome::ili9xxx::ILI9XXXST7796]+0x6c): undefined reference to `esphome::display::DisplayBuffer::write_display_data()'
collect2: error: ld returned 1 exit status
*** [.pioenvs/ha-touchscreen02/firmware.elf] Error 1

lukasz-tuz · July 4, 2023, 7:19pm

get this Display working?
=> 3.95寸86盒_深圳市启明智显科技有限公司

There’s no out-of-the-box way of using it in ESPHome, but seems that Arduino_GFX library supports it.

lukasz-tuz · July 4, 2023, 8:37pm

Thanks for checking this!
Inside HA, I don’t get it. Something is wrong with paths and where files end up. I’m going to have to check it inside my HA setup.
Outside HA, problem is that implementation of individual display drivers is not consistent. While st7789v that I’ve been using for development implements write_display_data() method, I’m guessing that ili9xxx does not. Supporting more than one display device is something I’m definitely going to have to work on.

DerPicknicker · July 5, 2023, 5:03am

Hi, thank you for the hint… Unfortunately esphome doesn’t support the display and also ArduinoGFX is not supported by esphome. It seems that you can add Arduino library’s but this is something which I have never did. So in the end I will give up, due my small amount of time. Hopefully the display IC get esphome support soon.

Yannik · January 23, 2024, 8:17am

I am new to ESPHome, but can it be used for this display? Unfortunately, I haven’t managed to get anything shown on the display yet…

DerekSN · January 29, 2024, 8:04pm

Hi

Yes you can use that display, the key is to set it to ili9342 and the resolution to 320 x 240, not 240 x 320

display:

platform: ili9xxx
model: ili9342
spi_id: tft
cs_pin: GPIO15
dc_pin: GPIO2
rotation: 0

Have a look at https://github.com/jonnybergdahl/ESP32-Cheap-Yellow-Display/tree/main/Examples/ESPHome this helped me out

Cheers

danhiking · June 29, 2024, 4:50pm

Glad I found this! Two questions; one for @DerekSN : I have the same display but connected to an early ESP32 that has no param. Therefore the display buffer allocation always fails. Do you know if there’s a way to disable the buffer and write directly to the display?

And for @lukasz-tuz , you mentioned that LVGL requires a display driver – when used with the esphome display driver, at what level does it hook up? Any way I can get around not having enough ram?

EDIT: I see that in the Component, LVGL operates on a buffer, so I’m not getting out of that.

I know the display has examples using LVGL and zero psram, so it must at least be possible. The only question is whether I can do it in esphome (which is really key for me.)

danhiking · June 30, 2024, 4:45pm

I just discovered that the new LVGL Graphics — ESPHome component () has a buffer_size option specifically for this kind of situation, when there is little or no PSRAM. That might be my way out.

DerekSN · July 1, 2024, 11:36pm

I am pretty sure LVGL requires a buffer, so being able to set this size should help get this working for you.

danhiking · July 2, 2024, 2:02pm

Following the use of the new esphome-lvgl component, I no longer get out-of-memory errors. However, ili9341 still doesn’t seem to respond.

I noticed that your config specifies cs_pin in addition to dc_pin. Do you know if that’s a requirement? I have this device, and I cannot for the life of me figure out what the dc_pin is.

Maybe it’s not that, but it’s odd that there’s no error AND no response from the screen, even though the LVGL logs show updates every second.

UPDATE: I found the CS pin on an obscure site, (pin 15), but the screen is still blank

esphome:
  platformio_options:
    build_flags: "-DBOARD_HAS_PSRAM"

esp32:
  board: esp32dev
  framework:
    # type: arduino
    type: esp-idf

# Enable logging
logger:
  level: VERBOSE
  logs:
    ili9xxx: VERBOSE
    

external_components:
 - source: github://clydebarrow/esphome@lvgl
   components: [ lvgl ] 

debug:
  update_interval: 5s

text_sensor:
  - platform: debug
    device:
      name: "Device Info"
    reset_reason:
      name: "Reset Reason"
  - platform: wifi_info
    ip_address:
      name: ESP IP Address
      id: ip_address
      on_value: 
        then:
          - lvgl.label.update:
              id: ip_address_label
              text:
                format: "%s"
                args: [ 'id(ip_address).get_state().c_str()' ]
    ssid:
      name: ESP Connected SSID
    bssid:
      name: ESP Connected BSSID
    mac_address:
      name: ESP Mac Wifi Address
    scan_results:
      name: ESP Latest Scan Results
    dns_address:
      name: ESP DNS Address

spi:
  - id: spi_bus0
    clk_pin: 18
    mosi_pin: 23
    miso_pin: 19

display:
  - platform: ili9xxx
    model: ili9341
    id: main_display
    dc_pin: 2
    spi_id: spi_bus0
    data_rate: 20MHz
    update_interval: never
    auto_clear_enabled: false
    invert_colors: true
    dimensions:
      height: 240
      width: 320
    transform:
      swap_xy: true 
      mirror_y: true
      mirror_x: true

lvgl:
  log_level: TRACE
  text_font: unscii_8
  align: center
  displays:
    - display_id: main_display
  buffer_size: 25%
  widgets:
    - obj:
        bg_color: 0xFF0000
        border_width: 8
        pad_all: 8
        height: size_content
        width: size_content
        widgets:
        - label:
            align: CENTER
            text: 'IP Address'
        - label:
            id: ip_address_label
            align: CENTER
            text: 'n/a'
        - label:
            id: time_label
            align: CENTER
            text: 'n/a'

script:
  - id: time_update
    then:
      - lvgl.label.update:
          id: time_label
          text:
            format: "%02d : %02d"
            args: [ 'id(time_comp).now().minute', 'id(time_comp).now().second' ]

time:
  - platform: homeassistant
    id: time_comp
    on_time_sync:
      - script.execute: time_update
    on_time:
      - minutes: '*'
        seconds: '*'
        then:
          - script.execute: time_update