ESPHome Ducky - MQTT keyboard/mouse control

Share your Projects!
1

image
image800×472 58.8 KB

What is it?
Control devices with a virtual usb keyboard/mouse.

How does it work?

  • The arduino is emulating HIDs (Human Interface Devices) such as keyboard/mouse via USB
  • The ESPHome board is receiving commands via wifi/mqtt and writes them to the serial port
  • The commands are in Ducky Script syntax
  • The commands are read from the serial port by the arduino and executed as keyboard/mouse actions
  • Furthermore it’s possible to get the host machine’s power state from the USB state via the serial port

For example, this will open up notepad and type some text in it:

service: mqtt.publish
data:
  topic: esphome/nodemcu/run
  payload: |
    GUI r
    DELAY 200
    STRING notepad.exe
    ENTER
    DELAY 200
    STRING Hello World

You can do basically everything that a keyboard and mouse can do. Media keys and power buttons are also supported. See github project for more information.
More examples:

2 Likes
2

I have just tested a rpi pico, which work out of the box as a HID device. My use case is slightly different, connecting physical buttons and a rotary encoder to a media player. You could join the pico to an esp to make a wifi device.

1 Like
3

I guess my arduino sketch wouldn’t work on the pico? But yeah, the concept should be applicable. Has the pico any major advantages?

4

The pico had the advantage of being

  1. small

  2. available within a few days in a country that goes to sleep during summer.

  3. it runs micropython and circuitpython which meant I could shamelessly use a howto I found on the net.

1 Like
5

Is there any chance that, if you use an esp32 that you’ll receive an command from esphome and then send a keystroke with ble to a ble connected host?

6

That seems like a solution for you: ESPHome BLE Keyboard

You can use the receive-command-part from my project and the send-part from that project.

2 Likes
7

Thank you! This is exactly what I need to wake up a PC from sleep which doesn’t have wake on LAN via the BIOS.

Can I just buy one of the Ducky built hardware dongles and flash over your project?

8

Hi, the device you’ve linked uses a different communication protocol (I²C instead of serial), so you would have to adjust that in both firmwares. I haven’t tried it myself though.

9

Really cool project, @jwm
What do you think of the idea to let the Bluetooth radio of a ESP32 emulate a Bluetooth keyboard directly to the computer?

I found this, so the emulation part could work, I guess. But don’t know if you can combine the code to use Bluetooth to the computer, and Wi-Fi to the Home Assistant server.

Edit: I now see that you wrote about something similar a few posts from the end.
But feel free to comment anyway. :slight_smile:

10

I tried that ble project and had some issues with android controls (power button IIRC) and some reconnect problems after sleep of the android-box. So I’m still using the arduino over usb.

In general I find NicoHood’s usb library (which my project uses) much more powerfull especially with special keys (even gamepad is possible) or different language layouts.

1 Like
11

Thanks for the reply. I’ll definitely build your project if the Bluetooth one isn’t stable enough.

12

hey, I know this is a late follow up, but did you have any success with the Wifi Duck?
I’m also looking to power on my laptop and wol does not work through my thunderbolt dock, so this looks like an interesting solution…

13

I successfully got this to work when plugged into my desktop. However when I plugged it into my tesmart KVM (HKS0202A2U) It wouldn’t pick up on the HID Keystrokes. Do you have any ideas why this didn’t work as expected?

14

Since it has hotkey switching, I guess that it’s not passing through the arduino. Instead it will interpret the keyboard commands itself first and mimic the commands to the pc?

The arduino board firmware does setup 4 hid devices in the code. Maybe the kvm switch cannot handle that. You can try to enable only the keyboard, maybe this will work.

EDIT: Here is a possibly related issue: KVM Switch (Disable CDC Serial for better BIOS/UEFi/Custom compatibility) · Issue #225 · NicoHood/HID · GitHub

15

:rocket: New Support: LILYGO T-PicoC3 & Raspberry Pi Pico (RP2040)

I’m excited to share a new experimental branch with support for additional hardware options:

:loudspeaker: New Branch: GitHub - jensweimann/esphome_ducky at lilygo-t-pico-c3

Added Support For:

  • LILYGO T-PicoC3 - An integrated solution with both ESP32-C3 and RP2040 on one board!
  • Raspberry Pi Pico (RP2040) - Can now be used as an alternative to the Arduino Pro Micro

Current Status:

  • :white_check_mark: Basic keyboard functionality works
  • :white_check_mark: Mouse controls function properly
  • :white_check_mark: Most common keys are working

Known Limitations:

  • :warning: Special characters like äöß are not working yet
  • :warning: USB suspend detection not implemented
  • :warning: Many key combinations remain untested

This implementation uses Earle Philhower’s Arduino core for RP2040 with its USB HID capabilities, which provides most functionality but isn’t as feature-complete as the NicoHood/HID stack.

Feel free to test and provide feedback! This is an experimental feature and contributions are welcome.

2 Likes
16

Thanks so much for the project. I have a computer with no Wake on LAN capability, so this is exactly what I was looking for.

I acquired a LILYGO Pico-C3 to give this a try. Some hiccups I came across:

  • When flashing the RP2040 side, I was missing the correct WIndows USB driver. Visual Studio Code would begin flashing to the COM port, disconnect, and the Pi would reconnect as “RP2 boot” in device manager, with no port. Using Zadiq, I was able to install the WindowsUSB driver to RP2 boot, then re-attempt the flash in Visual Studio Code.
  • When flashing the ESP side, I noticed there was an extra callout for encryption key, which caused issues (lines 66 and 67 are also 74 and 75 in lilygo-c3.yaml). Deleting lines 74 and 75 fixed this issue.
  • To properly flash the ESP side, you must short the IO9 pin to ground, otherwise ESPhome Web (or other flashing softwares) will give you an error. I stuck a wire into these two spots and held it in place to properly flash.

I’m now at the point where I’m ready to try everything, however I noticed the Github instruction is largely for mqtt, which I don’t believe applies here? Also, since the USB-C port on the LILYGO Pico-C3 is polar, I’m not sure how to power the ESP32 while also having the RP2040 plugged into the PC?

Thanks again for all the work on this!

17

Glad do see, that someone is using this project :sweat_smile:

Yes, you’re right, I didn’t adjust the manual yet. After flashing the esp32 and adding to your home assistant, you will get an action in HA named something like your_device.ducky_request and you can use it the same way as that script which encapsulates the mqtt topic in this example: esphome_ducky/homeassistant/ha_windows_control.yaml at 0a4d6184f67e812957f2ae052038f63176af52f5 · jensweimann/esphome_ducky · GitHub

You only let the rp2040 side connected to pc (blue led) since that board is doing the HID stuff. The esp32 board is sharing 5v power with rp2040 out of box.