Gesture controller via MQTT

Share your Projects!
1

Hi guys,
I would love to create a gesture controller based on APDS9960 module (cheap RGB, gesture module ~2$ on Aliexpress) and ESP8266.

The idea is simple. ESP8266 connects to MQTT via WIFI and reads states from APDS9960.
The states are: UP DOWN RIGHT LEFT NEAR FAR.

Using these we would be able to create a MQTT sensor which would be used for automation.

Everything seems to be there
ESP library for APDS9960 https://github.com/jonn26/SparkFun_APDS-9960_Sensor_Arduino_Library with examples

However I completely lack of knowledge of writiting a sketch, :blush:

I will try my best but yeah it will take years to create :smile:

Maybe someone experienced would be able to write such a sketch? and make this project alive?

4 Likes
2

Hey,

you could try to copy Bruh’s multisensor sketch as a basis and add the code needed for the chip.
You can watch this guy playing around with it…

Should be a fun project…

1 Like
3

Give it a shot, you’ll be shocked at how easy it is. Here’s all the code you need to send an mqtt message on a nodemcu

#include<PubSubClient.h>
#include<ESP8266WiFi.h>

void callback(char* topic, byte* payload, unsigned int length){
  
}
WiFiClient espClient;
const char* mqtt_server = "192.168.1.2";
PubSubClient client(mqtt_server, 1883, callback, espClient);
const char* ssid     = "your ssid";
const char* password = "wifi password"; 

void setup(){
 while (WiFi.status() != WL_CONNECTED) {
  delay(500);
  Serial.print(".");
}
Serial.print("connect. ip address: ");
Serial.println(WiFi.localIP());


}
void loop(){
  detect stuff
  if(!client.connected()){
    reconnect();
  }
  client.publish("topic/subtopic", "payload");
}
    void reconnect() {
      while (!client.connected()) {
        Serial.print("Attempting MQTT connection...");
        if (client.connect("ESP8266Client", "username", "password")) {
          Serial.println("connected");
        } else {
          Serial.print("failed, rc=");
          Serial.print(client.state());
          Serial.println(" try again in 5 seconds");
          delay(5000);
        }
      }
    }

replace ‘detect stuff’ with code for the board from the examples you posted and you’re done.

1 Like
4

I made it!

I read the sensor and ESP sends the commands to MQTT! :slight_smile:

I can control the HA with gestures IT IS SOOO COOL :slight_smile:

True Jedi :slight_smile:

Thanks guys

1 Like
5

Can you tell what you are controlling? My first idea was to control spotify. Up: play, down: pause, skipping songs and volume control with the directions.

Maybe I should order some myself. :slightly_smiling_face:

6

With the sensor you can send 6 commands. Witch means you can trigger 6 unique automation per device.

Right now as it is my first evening with it I just wrote a basic room light control. So left movement turns the lights off in the living room, right turns them on. Up turns on the bed room and down again turns it off.

Other 2 commands right now don’t have a use.

It for sure can be used to control a media center.

I will do some more testing and share my experience here.

7

Can you please share your code? Well done!!

8

@masterkenobi - you seen this? Right up your alley :sunglasses:

1 Like
9

Cool. Thanks for sharing this. Will definitely try it in future.

I’m wondering…

  1. How far can the sensor detects the gesture? Can it detects my gesture if I put the sensor on top of the TV while I’m sitting on my couch 3-4 meters away?
  2. Can it detects in the dark?
10

  1. It doesn’t work from that far. You have to wave your hand 2-3 cm from the sensor. So the battery version is advised to use as a remote controller.
    You can also attach the sensor to a lamp to control it instead of having a boring switch :wink:

  2. I haven’t checked if it works in darkness but l’ll do that later.

Also l’ll upload the sketch to github but firstly I need to revise it as I noticed that after a while the sensor stops to respond and needs to be restarted manually.

2 Likes
11

Could u also test if it would work behind plexiglass? Or spy mirror…well …would like to build this in my magic mirror project …which BTW will be posted here as soon as it’s finished

12

Hi guys,
here you can find the code: https://github.com/Flagun/Gesture_to_MQTT/

The code was taken from bruh’s repository and modified to work with ADPS-9960 module.

After uploading the sketch to ESP8266 it will connect to your WIFI and MQTT broker.
Then you can play with the sensor.

Known problems:
ESP8266 stops after some time and needs to be restarted manually to work again (I think it is due to the “NONE” gesture value that is included in the sketch - I’ll test when I have some free time).

I will answer your other questions soon.

Have fun!

2 Likes
13

AS soon AS u got rid of the known error …please let me know. I’d like integrating this in my mirror

1 Like
14

I gave it a 2nd try and it works stable for 24 hr now. No changes in the code.

15

What did u change then? Just unplugged and plugged again ?

16

Could You check if sensor works behind a glass? Will order one and test it if its working

17

@Flagun add a 1000uF/100uF capacitor on your 3.3V line, this sensor needs some serious juice and use a good power source for your 5V line to NodeMCU(esp8266).@thundergreen According to the docs, it is specifically made to be used behind dark glass (cough smartphone). @masterkenobi Since the light used to detect gesture is generated from the LED inside the sensor, your ambient light can be dark or bright.

I have to mention that it does color detection fairly well too, but remember to put a bright source of white light (eg, LEDs) next to the sensor. This is important for color sensing as the object usually covers the sensor and no light is reflected.

Also people ordering on ebay and Aliexpress, be sure to order the expensive APDS 9960 instead of cheap APDS9930 as the sellers intentionally interchangeably use keywords. They are not the same sensors, APDS9930 will not sense gesture.

1 Like
18

Thanks debsahu, I’ll power up the sensor with 5V. I hope it’ll eliminate the not-responding problem.

19

No don’t do that, your sda and scl must be at the same 5v level and Max voltage for APDS9960 is 3.8V. Use a cap instead.

20

Ok thanks again!