Working ESPHome config for ttgo ESP32 camera board with microphone

So the website shows 3 pins for the microphone, SCK, WS, SDO. Based on this I think it’s a digital microphone that uses I2S. I don’t see a component for this in ESPHome so I don’t think it’s currently supported. There are examples of this for arduino/ESPs online so it can, of course, be added by using existing libraries and modifying these examples. This is “on my todo list” but to be honest it’s a pretty long list so we’ll see if I get to it before someone else does.

I have quite a few ESP based DYI “things” I’ve built now and I really feel like I need to take the packaging to the next level and make things look more professional. It seems I am going to have to tackle 3D printing before I worry about microphone support.

1 Like

I realized after posting this that the camera image is flipped horizontally. This config fixes that:

substitutions:
  devicename: ttgocam
  friendly_name: Doorbell
  ip_address: 192.168.2.251

esphome:
  name: $devicename
  platform: ESP32
  #board: esp-wrover-kit
  board: esp32dev

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    static_ip: $ip_address
    gateway: 192.168.2.1
    subnet: 255.255.255.0
# Enable logging
logger:

# Enable Home Assistant API
api:
ota:
binary_sensor:
  - platform: gpio
    pin: 
      number: GPIO19
      mode: INPUT
      inverted: False
    name: $friendly_name Motion
    device_class: motion
    filters:
      - delayed_off: 1s
  - platform: gpio
    pin:
      number: GPIO0
      mode: INPUT_PULLUP
      inverted: True
    name: $friendly_name Button
    filters:
      - delayed_off: 50ms
  - platform: status
    name: $friendly_name Status
sensor:
  - platform: wifi_signal
    name: $friendly_name WiFi Signal
    update_interval: 10s
  - platform: uptime
    name: $friendly_name Uptime
esp32_camera:
  name: $friendly_name Camera
  external_clock:
    pin: GPIO4
    frequency: 20MHz
  i2c_pins:
    sda: GPIO18
    scl: GPIO23
  data_pins: [GPIO34, GPIO13, GPIO14, GPIO35, GPIO39, GPIO12, GPIO15, GPIO36]
  vsync_pin: GPIO5
  href_pin: GPIO27
  pixel_clock_pin: GPIO25
 # power_down_pin: GPIO26
  resolution: 640x480
  jpeg_quality: 10
  vertical_flip: true
  horizontal_mirror: true
i2c:
  sda: GPIO21
  scl: GPIO22
font:
  - file: "fonts/times-new-roman.ttf"
    id: tnr1
    size: 20
  - file: "fonts/times-new-roman.ttf"
    id: tnr2
    size: 24
time:
  - platform: homeassistant
    id: homeassistant_time
display:
  - platform: ssd1306_i2c
    model: "SSD1306 128x64"
    address: 0x3C
    rotation: 180
    lambda: |-
      it.strftime(64, 0, id(tnr1), TextAlign::TOP_CENTER,"%m-%d-%Y", id(homeassistant_time).now());
      it.strftime(64, 64, id(tnr2), TextAlign::BASELINE_CENTER, "%I:%M:%S%p", id(homeassistant_time).now());
3 Likes

Hi. Thank you for posting this.
It’s working great.
Where do I put the font file, to make the configuration compile?

Kind regards,
Esben

It’s been quite a while since I set this up . . . I just took a look and I see two copies of the file. One is in the same directory as the yaml file and the other is in a /fonts subdirectory. I’m not sure which one is correct.

BobG, do you have any ideas for getting the microphone to work? I was thinking of scouring github for some python packages that may work and contacting the manufacturer. Any other ideas?

I haven’t looked into this. My initial thoughts would be to find a c++ example of using a microphone on an esp32 board and incorporating that into an ESPHome custom sensor.

BobG, good idea. Let me see if I can find that - will get back to you asap.

Thanks for sharing. My TTGO-Cam (purchased in July 2020 from aliexpress) has a different pin allowcation. I’ve posted my conifg below, hope it saves somebody some time.

substitutions:
  friendly_name: Doorbell ESP

esphome:
  name: doorbellcam
  platform: ESP32
  board: esp-wrover-kit

wifi:
  ssid: "urssid"
  password: "urpaswrd"

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Doorbellcam Fallback Hotspot"
    password: "urfbpaswrd"

captive_portal:

# Enable logging
logger:

# Enable Home Assistant API
api:

ota:

binary_sensor:
  - platform: gpio
    pin: 
      number: GPIO19
      mode: INPUT
      inverted: False
    name: $friendly_name Motion
    device_class: motion
    filters:
      - delayed_off: 1s
  - platform: gpio
    pin:
      number: GPIO15
      mode: INPUT_PULLUP
      inverted: True
    name: $friendly_name Button
    filters:
      - delayed_off: 50ms
  - platform: status
    name: $friendly_name Status
sensor:
  - platform: wifi_signal
    name: $friendly_name WiFi Signal
    update_interval: 10s
  - platform: uptime
    name: $friendly_name Uptime
esp32_camera:
  name: $friendly_name Camera
  external_clock:
    pin: GPIO4
    frequency: 20MHz
  i2c_pins:
    sda: GPIO18
    scl: GPIO23
  data_pins: [GPIO34, GPIO13, GPIO14, GPIO35, GPIO39, GPIO38, GPIO37, GPIO36]
  vsync_pin: GPIO5
  href_pin: GPIO27
  pixel_clock_pin: GPIO25
  resolution: 640x480
  jpeg_quality: 10
  vertical_flip: true
  horizontal_mirror: true
i2c:
  sda: GPIO21
  scl: GPIO22
font:
  - file: "fonts/times-new-roman.ttf"
    id: tnr1
    size: 20
  - file: "fonts/times-new-roman.ttf"
    id: tnr2
    size: 24
time:
  - platform: homeassistant
    id: homeassistant_time
display:
  - platform: ssd1306_i2c
    model: "SSD1306 128x64"
    address: 0x3C
    rotation: 180
    lambda: |-
      it.strftime(64, 0, id(tnr1), TextAlign::TOP_CENTER,"%m-%d-%Y", id(homeassistant_time).now());
      it.strftime(64, 64, id(tnr2), TextAlign::BASELINE_CENTER, "%I:%M:%S%p", id(homeassistant_time).now());
4 Likes

Thanks for sharing and help!

3 Likes

Thanks for your help!
I had your same configuration but also needed to modify main.cpp code for it to work as specified on another thread.
Just in case anyone else happens to have the same issue.

Hey Dturgel, were you ever able to get the microphone working please?

Thanks

Hi I have problems getting the camera to work. Apparently I have a V1.6.2 version of the board. If I compile the sketch without the camera part everything works nicely. However If I add

esp32_camera:
  name: $friendly_name Camera
  external_clock:
    pin: GPIO4
    frequency: 20MHz
  i2c_pins:
    sda: GPIO18
    scl: GPIO23
  data_pins: [GPIO34, GPIO13, GPIO14, GPIO35, GPIO39, GPIO38, GPIO37, GPIO36]
  vsync_pin: GPIO5
  href_pin: GPIO27
  pixel_clock_pin: GPIO25
  jpeg_quality: 10
  vertical_flip: true
  horizontal_mirror: false

I cannot even connect to the board anymore via wifi. Although OTA works.

The problem seems to be the missing pullup for GPIO14.
Therefore I added these lines:

binary_sensor:
  - platform: gpio
    name: DummyIO14
    internal: true # sensor will not be exposed to Home Assistant
    pin:
      number: GPIO14
      mode: INPUT_PULLUP

Good luck :+1:
Instead of name: and internal: you can simply use id: DummyIO14 and the sensor will also not be exposed to Home Assistant.

Thanks! this is great - no more errors. I managed to get the board live without this addition once and have been struggling to get it to work for a week. Was prepared to bin it thinking I bricked it. @Mikrofarad - one question though, is it normal that the dummy sensor updates its status continuously? Any way to reduce the frequency?

@PaulAntonDeen fyi

Sorry, I don’t know any solution. What I did is changing the logging level:

logger:
  level: Error  # During development set it to Debug

You can use a filter:

  - platform: gpio
    id: dummy
    pin:
      number: GPIO14
      mode: INPUT_PULLUP
    filters:
      - delayed_on: 1000ms
1 Like

Hi All, I have the same or similar TTGO-Camera, and just got it working with the help from your above comments - thanks lots all <3

Mine is on version 1.7 though, and totally different GPIO pin config. Please see my config below for reference in case it helps anyone else <3

(I’ve hashed out secrets and changed name, ip addr, and printing text for security reasons) :wink:

esphome:
name: multifunctiondevice01

substitutions:
friendly_name: MF01

esp32:
board: esp32dev

Enable logging

logger:

Enable Home Assistant API

api:
encryption:
key: “whatevers”

ota:
password: “whatevers”

wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
manual_ip:
static_ip: 10.10.10.2
gateway: 10.10.10.1
subnet: 255.255.255.0

binary_sensor:

  • platform: gpio
    pin:
    number: GPIO33
    mode: INPUT
    inverted: False
    name: Motion
    device_class: motion
    filters:
    • delayed_off: 1s
  • platform: gpio
    pin:
    number: GPIO34
    #mode: INPUT_PULLUP
    mode: INPUT
    inverted: True
    name: $friendly_name Button
    filters:
    • delayed_off: 50ms
  • platform: gpio
    id: dummy
    pin:
    number: GPIO14
    mode: INPUT_PULLUP
    filters:
    • delayed_on: 1000ms
  • platform: status
    name: $friendly_name Status
    sensor:
  • platform: wifi_signal
    name: $friendly_name WiFi Signal
    update_interval: 10s
  • platform: uptime
    name: $friendly_name Uptime
    esp32_camera:
    name: $friendly_name Camera
    external_clock:
    pin: GPIO32
    frequency: 20MHz
    i2c_pins:
    sda: GPIO13
    scl: GPIO12
    data_pins: [GPIO5, GPIO14, GPIO4, GPIO15, GPIO18, GPIO23, GPIO36, GPIO39]
    vsync_pin: GPIO27
    href_pin: GPIO25
    pixel_clock_pin: GPIO19

power_down_pin: GPIO26

resolution: 640x480
jpeg_quality: 10
vertical_flip: true
horizontal_mirror: true
i2c:
sda: GPIO21
scl: GPIO22
font:

  • file: “fonts/times-new-roman.ttf”
    id: tnr1
    size: 18
  • file: “fonts/times-new-roman.ttf”
    id: tnr2
    size: 24
    time:
  • platform: homeassistant
    id: homeassistant_time
    display:
  • platform: ssd1306_i2c
    model: “SSD1306 128x64”
    address: 0x3C
    rotation: 180
    lambda: |-
    it.strftime(64, 0, id(tnr1), TextAlign::TOP_CENTER,“My House”, id(homeassistant_time).now());
    it.strftime(64, 32, id(tnr2), TextAlign::CENTER,“Go Away!”, id(homeassistant_time).now());
    it.strftime(64, 64, id(tnr1), TextAlign::BASELINE_CENTER, “%H:%M:%S”, id(homeassistant_time).now());

ok, apologies for the above, the formatting got ruined by this site X-)
you’ll just need to look at the original posts and apply my GPIO pin changes (there’s a lot of them, as basically they nearly all change!) :slight_smile: Hope this helps <3

Sooo… anyone ever got the Microphone to work?
V1.6.2 with Mic is supposed to have SCK26 WS32 SDO33

i tried with this yaml:


esphome:
  name: esphome-web-454f70
  friendly_name: Camera-ESP

esp32:
  board: esp32dev
  framework:
    type: arduino

i2c:
  sda: GPIO21
  scl: GPIO22

i2s_audio:
  i2s_lrclk_pin: GPIO32
  i2s_bclk_pin: GPIO26

microphone:
  - platform: i2s_audio
    id: adc_mic
    i2s_din_pin: GPIO33
    adc_type: external
    pdm: false

voice_assistant:
  microphone: adc_mic

time:
  - platform: homeassistant
    id: homeassistant_time

binary_sensor:
  - platform: gpio
    pin: GPIO19
    name: "ttgocam2 PIR"
    device_class: motion

  - platform: gpio
    pin:
      number: GPIO15
      mode: INPUT_PULLUP
      inverted: True
    name: "ttgocam2 Button"

sensor:
  - platform: wifi_signal
    name: "ttgocam2 WiFi Signal"
    update_interval: 60s
font:
  - file: "fonts/times-new-roman.ttf"
    id: tnr1
    size: 20
  - file: "fonts/times-new-roman.ttf"
    id: tnr2
    size: 35

display:
  - platform: ssd1306_i2c
    model: "SSD1306 128x64"
    address: 0x3C
    lambda: |-
      it.strftime(64, 0, id(tnr1), TextAlign::TOP_CENTER,"%d-%m-%Y", id(homeassistant_time).now());
      it.strftime(64, 64, id(tnr2), TextAlign::BASELINE_CENTER, "%H:%M", id(homeassistant_time).now());
esp32_camera:
  external_clock:
    pin: GPIO4
    frequency: 20MHz
  i2c_pins:
    sda: GPIO18
    scl: GPIO23
  data_pins: [GPIO34, GPIO13, GPIO14, GPIO35, GPIO39, GPIO38, GPIO37, GPIO36]
  vsync_pin: GPIO5
  href_pin: GPIO27
  pixel_clock_pin: GPIO25
  jpeg_quality: 10
  vertical_flip: true
  horizontal_mirror: false

  # Image settings
  name: My Camera



# Enable logging
logger:

but i don’t know how to integrate that into HA now… or if it even works?
The Bootlog from the ESP32 doesn’t mention audio or microphone anywhere…

[03:29:01][I][app:102]: ESPHome version 2023.11.6 compiled on Dec  6 2023, 03:28:16
[03:29:01][C][wifi:559]: WiFi:
[03:29:01][C][wifi:391]:   Local [redacted]
[03:29:01][C][wifi:396]:   SSID: [redacted]
[03:29:01][C][wifi:397]:   IP Address: 192.168.1.236
[03:29:01][C][wifi:399]:   BSSID: [redacted]
[03:29:01][C][wifi:400]:   Hostname: 'esphome-web-454f70'
[03:29:01][C][wifi:402]:   Signal strength: -56 dB ▂▄▆█
[03:29:01][C][wifi:406]:   Channel: 12
[03:29:01][C][wifi:407]:   Subnet: 255.255.255.0
[03:29:01][C][wifi:408]:   Gateway: 192.168.1.1
[03:29:01][C][wifi:409]:   DNS1: 192.168.1.1
[03:29:01][C][wifi:410]:   DNS2: 0.0.0.0
[03:29:01][C][logger:416]: Logger:
[03:29:01][C][logger:417]:   Level: DEBUG
[03:29:01][C][logger:418]:   Log Baud Rate: 115200
[03:29:01][C][logger:420]:   Hardware UART: UART0
[03:29:01][C][i2c.arduino:053]: I2C Bus:
[03:29:01][C][i2c.arduino:054]:   SDA Pin: GPIO21
[03:29:01][C][i2c.arduino:055]:   SCL Pin: GPIO22
[03:29:01][C][i2c.arduino:056]:   Frequency: 50000 Hz
[03:29:01][C][i2c.arduino:059]:   Recovery: bus successfully recovered
[03:29:01][I][i2c.arduino:069]: Results from i2c bus scan:
[03:29:01][I][i2c.arduino:075]: Found i2c device at address 0x3C
[03:29:01][C][gpio.binary_sensor:015]: GPIO Binary Sensor 'ttgocam2 PIR'
[03:29:01][C][gpio.binary_sensor:015]:   Device Class: 'motion'
[03:29:01][C][gpio.binary_sensor:016]:   Pin: GPIO19
[03:29:01][C][gpio.binary_sensor:015]: GPIO Binary Sensor 'ttgocam2 Button'
[03:29:01][C][gpio.binary_sensor:016]:   Pin: GPIO0
[03:29:01][D][api:102]: Accepted 192.168.1.1
[03:29:01][C][homeassistant.time:010]: Home Assistant Time:
[03:29:01][C][homeassistant.time:011]:   Timezone: 'CET-1CEST,M3.5.0,M10.5.0/3'
[03:29:01][C][esp32_camera:048]: ESP32 Camera:
[03:29:01][C][esp32_camera:049]:   Name: My Camera
[03:29:01][C][esp32_camera:050]:   Internal: NO
[03:29:01][C][esp32_camera:052]:   Data Pins: D0:34 D1:13 D2:14 D3:35 D4:39 D5:38 D6:37 D7:36
[03:29:01][C][esp32_camera:053]:   VSYNC Pin: 5
[03:29:01][C][esp32_camera:054]:   HREF Pin: 27
[03:29:01][C][esp32_camera:055]:   Pixel Clock Pin: 25
[03:29:01][C][esp32_camera:056]:   External Clock: Pin:4 Frequency:20000000
[03:29:01][C][esp32_camera:060]:   I2C Pins: SDA:18 SCL:23
[03:29:01][C][esp32_camera:062]:   Reset Pin: -1
[03:29:01][C][esp32_camera:080]:   Resolution: 640x480 (VGA)
[03:29:01][C][esp32_camera:129]:   JPEG Quality: 10
[03:29:01][C][esp32_camera:131]:   Contrast: 0
[03:29:01][C][esp32_camera:132]:   Brightness: 0
[03:29:01][C][esp32_camera:133]:   Saturation: 0
[03:29:01][C][esp32_camera:134]:   Vertical Flip: ON
[03:29:01][C][esp32_camera:135]:   Horizontal Mirror: OFF
[03:29:01][C][esp32_camera:136]:   Special Effect: 0
[03:29:01][C][esp32_camera:137]:   White Balance Mode: 0
[03:29:01][C][esp32_camera:140]:   Auto Exposure Control: 1
[03:29:01][C][esp32_camera:141]:   Auto Exposure Control 2: 0
[03:29:01][C][esp32_camera:142]:   Auto Exposure Level: 0
[03:29:01][C][esp32_camera:143]:   Auto Exposure Value: 300
[03:29:01][C][esp32_camera:144]:   AGC: 1
[03:29:01][C][esp32_camera:145]:   AGC Gain: 0
[03:29:01][C][esp32_camera:146]:   Gain Ceiling: 0
[03:29:01][C][esp32_camera:152]:   Test Pattern: NO
[03:29:01][D][api.connection:1089]: Home Assistant 2023.11.3 (192.168.1.1): Connected successfully
[03:29:01][C][psram:020]: PSRAM:
[03:29:01][C][psram:021]:   Available: YES
[03:29:01][C][psram:024]:   Size: 4095 KB
[03:29:01][D][time:045]: Synchronized time: 2023-12-06 03:29:01
[03:29:01][C][ssd1306_i2c:023]: I2C SSD1306
[03:29:01][C][ssd1306_i2c:023]:   Rotations: 0 °
[03:29:01][C][ssd1306_i2c:023]:   Dimensions: 128px x 64px
[03:29:01][C][ssd1306_i2c:024]:   Address: 0x3C
[03:29:01][C][ssd1306_i2c:025]:   Model: SSD1306 128x64
[03:29:01][C][ssd1306_i2c:027]:   External VCC: NO
[03:29:01][C][ssd1306_i2c:028]:   Flip X: YES
[03:29:01][C][ssd1306_i2c:029]:   Flip Y: YES
[03:29:01][C][ssd1306_i2c:030]:   Offset X: 0
[03:29:01][C][ssd1306_i2c:031]:   Offset Y: 0
[03:29:01][C][ssd1306_i2c:032]:   Inverted Color: NO
[03:29:01][C][ssd1306_i2c:033]:   Update Interval: 1.0s
[03:29:01][C][captive_portal:088]: Captive Portal:
[03:29:01][C][mdns:115]: mDNS:
[03:29:01][C][mdns:116]:   Hostname: esphome-web-454f70
[03:29:01][C][ota:097]: Over-The-Air Updates:
[03:29:01][C][ota:098]:   Address: esphome-web-454f70.local:3232
[03:29:01][C][api:139]: API Server:
[03:29:01][C][api:140]:   Address: esphome-web-454f70.local:6053
[03:29:01][C][api:142]:   Using noise encryption: YES
[03:29:02][W][component:214]: Component ssd1306_base took a long time for an operation (0.27 s).
[03:29:02][W][component:215]: Components should block for at most 20-30ms.
[03:29:02][C][wifi_signal.sensor:009]: WiFi Signal 'ttgocam2 WiFi Signal'
[03:29:02][C][wifi_signal.sensor:009]:   Device Class: 'signal_strength'
[03:29:02][C][wifi_signal.sensor:009]:   State Class: 'measurement'
[03:29:02][C][wifi_signal.sensor:009]:   Unit of Measurement: 'dBm'
[03:29:02][C][wifi_signal.sensor:009]:   Accuracy Decimals: 0
[03:29:03][W][component:214]: Component ssd1306_base took a long time for an operation (0.26 s).
[03:29:03][W][component:215]: Components should block for at most 20-30ms.

i do see 7 devices.


Should the assist Pipeline simply get all the mic input? how would one check?
Or is my config not giving any output?
Is there a way to stream Mic Input to browser playback or such?

Okay… i got a little closer…

using Push to Talk function:

[04:04:48][D][binary_sensor:036]: 'ttgocam2 Button': Sending state ON
[04:04:48][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[04:04:48][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[04:04:48][D][voice_assistant:124]: microphone not running
[04:04:48][D][voice_assistant:206]: Requesting start...
[04:04:48][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[04:04:48][D][voice_assistant:443]: Client started, streaming microphone
[04:04:48][D][voice_assistant:422]: State changed from STARTING_PIPELINE to START_MICROPHONE
[04:04:48][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[04:04:48][D][voice_assistant:159]: Starting Microphone
[04:04:48][D][voice_assistant:422]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[04:04:48][D][esp-idf:000]: E (107291) I2S: i2s_dma_intr_init(538): Register I2S Interrupt error

[04:04:48][D][esp-idf:000]: E (107303) I2S: i2s_driver_install(1950): I2S interrupt initailze failed

WARNING esphome-web-454f70: Connection error occurred: [Errno 104] Connection reset by peer
INFO Processing unexpected disconnect from ESPHome API for esphome-web-454f70
WARNING Disconnected from API
INFO Successfully connected to esphome-web-454f70 in 0.004s

seems like the microphone is NOT running.

Edit:
This Guy seem to have same issue:

And this guy seems to have found the Solution to this Problem: