Vehicle detection with D1 Mini and magnetometer for gate actuation

Wanted to share a project I’ve been working on for a couple of weeks (lots of banggood waiting time :grin:)

tl;dr: D1 Mini + ESPHome + magnetometer detects vehicle wanting to exit closed gate and triggers it to open.

My folks have a large metal gate at the entrance of their driveway with a keypad on the outside. Guests can come easily, but due to an auto close after a few minutes it has been an unfortunately manual process to let them exit either by push button on the inside of the gate itself or a garage remote with frustratingly limited range.

They were interested in a vehicle detection induction loop but after looking at the price of easily $150+, I knew I could do it for cheaper and have it done smarter. [enter the D1 mini…]

I’ve previously deployed a D1 Mini Pro (pro for the external antenna) to give them the luxury of a gate opener app with some basic telemetry (open/closed data only). This would be the starting point for my custom vehicle detector. I originally considered an ultrasonic sensor to detect presence but decided to forgo this route to have a cleaner (and hopefully more accurate) detection. After some basic research I came up mostly empty handed but stumbled on a forum suggesting the use of a QMC5883L magnetometer. $7 and a manual build of ESPHome later I had a working prototype!

As I’d suspected, when a vehicle or other large ferrous object (or even a hand placed closely) passed above the magnetometer, its readings would change significantly. Basic tests proved that I would be able to use this for my detection. After some hacking in ESPHome I eventually landed on creating a sensor with a 15 second moving average of heading readings, and triggering my relay if the current heading reading varied from the average for more than 3 seconds. Some additional logic was introduced to filter out sporadic bad readings and prevent gate from actuating too often (or closing on a car passing through!).

The below is what I came up with:

  • Create sensor with average of last 15 heading headings (taken at 1s intervals)
  • Create “automation safety” binary input to prevent rapid firing of gate open from readings
  • If current reading varies ± 2.5° from average turn on internal binary sensor
  • If binary sensor on for more than 3 seconds AND gate is closed AND our “automation safety” binary sensor is off: trigger the relay and turn automation safety on for 120s

In the above we have ensured that:

  1. Something (presumably a large metal object) has caused a change in heading and has stayed for some time
  2. The gate is closed and probably needs opening
  3. We won’t accidentally close the gate on a car passing through. (The average heading readings will start to move toward the new reading to the cars magnetic field manipulation, and will change again when the car moves)

Here’s the full ESPHome code:

esphome:
  name: gate_actuator
  platform: ESP8266
  board: d1_mini_pro
  on_boot:
    then:
      # Set/get initial states
      - binary_sensor.template.publish:
          id: automation_safety
          state: OFF
      - binary_sensor.template.publish:
          id: heading_comparison
          state: OFF
      - component.update: battery_level

web_server:
  port: 80

logger:
  level: NONE
  
wifi:
  networks:
  - ssid: SSID1
    password: Pass1
  - ssid: SSID2
    password: Pass2

ota:
  safe_mode: True
  password: esphome_recovery

i2c:
  sda: D2
  scl: D1
  scan: False

script:
  - id: gate_toggle
    then:
      if:
        condition:
          and:
            # Make sure we aren't triggering too rapidly
            - binary_sensor.is_off: automation_safety
            # Make sure gate is closed or else we won't trigger
            - binary_sensor.is_off: gate_status
        then: 
          - binary_sensor.template.publish:
              id: automation_safety
              state: ON
          - switch.toggle: gate
          - delay: 120s
          - binary_sensor.template.publish:
              id: automation_safety
              state: OFF

mqtt:
  broker: VPS_IP
  username: user
  password: pass
  log_topic:
  # Allow the app to toggle the gate independent from any logic
  on_message:
    - topic: cmd/gate
      then:
        - switch.toggle: gate
  
sensor:
  - platform: qmc5883l
    address: 0x0D
    heading:
      id: heading
    data_rate: 10Hz
    range: 200uT
    oversampling: 512x
    update_interval: 1s

  # Template sensor to keep average of last 15 heading
  # readings so we can detect significant change
  - platform: template
    id: heading_average
    update_interval: 1s
    unit_of_measurement: "°"
    lambda: return id(heading).state;
    filters:
      - sliding_window_moving_average:
          window_size: 15
          send_every: 1
    # only trigger gate if heading comparison has been
    # true for more than 5s and safety is off
    on_value:
      then:
        if:
          condition:
            and:
              - for:
                  time: 3s
                  condition:
                    binary_sensor.is_on: heading_comparison
              # This is intentionally duplicated
              - binary_sensor.is_off: automation_safety
          then:
            - script.execute: gate_toggle
  
  - platform: adc
    pin: A0
    id: battery_level
    name: "Gate Battery Voltage"
    update_interval: 60s
    accuracy_decimals: 4
    filters:
      - multiply: 15.43

  - platform: uptime
    name: "Uptime"

binary_sensor:
  # Sensor to keep script from triggering in rapid succession
  - platform: template
    name: "Automation Safety"
    id: automation_safety

  # This is what is used to trigger the automation
  - platform: template
    name: Heading Comparison Trigger
    id: heading_comparison
    # Set our trigger to true if heading varies from avg more than 2.5deg 
    lambda: |-
      if (id(heading).state > id(heading_average).state + 2.5) {
        return true;
      }
      if (id(heading).state < id(heading_average).state - 2.5) {
        return true;
      }
      else {
        return false;
      }

  # Magnet sensor to get real-world state
  - platform: gpio
    pin:
      number: D7
      mode: INPUT_PULLUP
    name: Gate Status
    id: gate_status
    device_class: door
    filters:
      - delayed_on: 10ms

switch:
  - platform: shutdown
    name: "Shut Down"
  # Actual GPIO output
  - platform: gpio
    id: relay
    pin:
      number: D8
      inverted: False
  
  # Switch which toggles the relay on and off
  - platform: template
    id: gate
    name: "Gate Remote"
    icon: "mdi:gate"
    turn_on_action:
    - switch.turn_on: relay
    - delay: 500ms
    - switch.turn_off: relay
    - binary_sensor.template.publish:
        id: automation_safety
        state: ON
    - delay: 60s
    - binary_sensor.template.publish:
        id: automation_safety
        state: OFF

text_sensor:
  - platform: wifi_info
    ip_address:
      name: "IP Address"

In addition to vehicle detection there is a gate status sensor (open/closed), battery sensor (gate and microcontroller run on 12V battery), and functionality to open/close gate via app/mqtt. I wanted to be sure this could operate independently (I run Home Assistant at home but they live far away) and have built it so that no external connectivity is required but if present will add additional functionality.

Testing with magnetometer on end of long (30’) cat5 to ensure functionality and measure battery consumption:

How the QMC5883L was connected to Cat5:

Magnetometer mounted inside Sonoff IP66 junction box (this was buried underground inside 4" PVC for protection under rock driveway. Moisture absorbing packets optional, hopefully not needed! :laughing:):

Testing voltage divider for ADC. This later mated (upside down) to a D1 Proto Shield:

Completed “front” of stack:

Completed “back” of stack:

Wired up in gate electronics box:

Reed switch on swing arm to detect open/closed:

Wiring diagram:

Telemetry data and gate toggle Home Assistant:

Home Assistant Config:

sensor:
  - platform: mqtt
    state_topic: "gate_actuator/sensor/gate_battery_voltage/state"
    name: gate_battery_voltage
    unit_of_measurement: 'V'
  - platform: mqtt
    state_topic: "gate_actuator/sensor/ip_address/state"
    name: gate_ip_address
  - platform: mqtt
    state_topic: "gate_actuator/sensor/uptime/state"
    name: gate_uptime
    unit_of_measurement: 's'
  - platform: mqtt
    state_topic: "gate_actuator/status"
    name: gate_online_offline


binary_sensor:
  - platform: mqtt
    state_topic: "gate_actuator/binary_sensor/gate_status/state"
    name: gate_status
    device_class: door

  - platform: mqtt
    state_topic: "gate_actuator/binary_sensor/heading_comparison_trigger/state"
    name: gate_heading_comparison

  - platform: mqtt
    state_topic: "gate_actuator/binary_sensor/automation_safety/state"
    name: gate_automation_safety


automation:
  - alias: "Gate Heading Comparison Action"
    initial_state: 'on'
    trigger:
      - platform: state
        entity_id: binary_sensor.gate_automation_safety
        to: 'on'
    action:
      - service: notify.pushover
        data_template:
          message: >
            Vehicle Sensor Automation Fired!

  - alias: "Gate Battery Safety Notify"
    initial_state: 'on'
    trigger:
      - platform: numeric_state
        entity_id: sensor.gate_battery_voltage
        below: 8
    action:
      - service: notify.pushover
        data_template:
          message: >
            Battery Low: {{ states.sensor.gate_battery_voltage.state }}
          data:
            priority: 1

  - alias: "Webhook Trigger"
    initial_state: 'on'
    trigger:
      platform: webhook
      webhook_id: !secret gate_webhook_id
    action:
      service: script.turn_on
      entity_id: script.gate_toggle


script:
  mom_and_dad_gate_toggle:
    sequence:
      - service: mqtt.publish
        data:
          topic: "cmd/gate"
          payload: 'toggle'

Component Cost:

D1 Mini Pro: $4
D1 Mini Dual Base: $1.00
D1 Mini Relay Shield: $1.20
D1 Mini DC Power Shield: $1.40
D1 Mini Proto Board: $0.30
QMC5883L Triple Axis Compass Magnetometer: $6.59
Sonoff IP66 Junction Box: $6.99
Voltage Divider (to measure up to 25v): $1.38
Misc. cables: $0

Total: ~$22.96 USD.

Final thoughts:

This build works just as I’d expect, I’m very pleased with how it turned out. I do have a couple of concerns:

  1. The cat5 I’ve used is only plenum rated (if that). I didn’t bury it in any conduit so there’s a decent chance that it will degrade, ingest water, etc. We’ll see if this has near-term affects on the function, in the long term it should almost certainly be laid in a conduit, time will tell.
  2. The D1 Mini has a voltage divider on A0 already. It feels weird to stack the voltage dividers and my reading*15.43 was the result of some basic circuit diagram and real-world testing. In addition, the voltage sensor is not currently functioning as I’d expect (see the 0.2260V reading in the HomeAssistant screenshot?). I suspect the cable on the battery terminal is loose. In hindsight I should have soldered the sensor wire onto a washer to be tight alongside the other wiring on the battery + terminal. I also didn’t attach the GND to the voltage divider. I don’t think it’s necessary since the whole system is battery powered and should share the same ground but I’m not an electrical engineer. The D1 pulls power from the gate circuit board’s +12v and GND terminals.
  3. I’m not sure how well the IP66 Sonoff box will do underground in the rock driveway long term. The PVC pipe was a last minute addition and probably the saving grace. Even still I don’t know if the plexiglass cover on the junction box will eventually get cracked from pressure or have water seep in by another means.

I know this was a wall of text, thanks for reading! Questions? Suggestions?

24 Likes

Nice write-up, thanks for sharing !

1 Like

Thank you for the write-up. I am new to HomeAssistant and ESPHome … this is an invaluable resource and I will definitely use many of the components and configurations you described here.

1 Like

Great writeup!

I’ve seen some of the on_boot code used but I haven’t seen any clear explanation for it. Some questions if you can help me understand your ESPHome yaml:

  1. Does the below code create/initilize the two binary sensors for automation_safety and heading_comparison?
  2. Can you also use it to create input_booleans or input_numbers?
  3. And regarding the component.update for battery_level, what does this do in this part of the code?
  4. It looks like that it’s tied to the ADC sensor. Wouldn’t the ADC sensor still function fine without - component.update: battery_level? Or does this just force it to happen on boot?
  on_boot:
    then:
      # Set/get initial states
      - binary_sensor.template.publish:
          id: automation_safety
          state: OFF
      - binary_sensor.template.publish:
          id: heading_comparison
          state: OFF
      - component.update: battery_level

Also, have you tested how big the detection area is? Wondering if you can use this to pickup a gocart in a driveway or a bicycle?

  1. Does the below code create/initilize the two binary sensors for automation_safety and heading_comparison?

No, the on_boot sets an initial value, the sensors have to be defined and “created” in the binary_sensor section first. Full disclosure, these on_boot state settings may not actually be required. In all honesty I haven’t run it without them to see what the values come up as.

  1. Can you also use it to create input_booleans or input_numbers?

This doesn’t create the sensors, just sets a value for them.

  1. And regarding the component.update for battery_level, what does this do in this part of the code?

The code I’ve posted is slightly modified. I’m currently attempting to troubleshoot the adc (battery level) so I’ve lowered the update time to 60s. Once I figure out the problem I’ll change that update time to 600 seconds. The on_boot just forces a reading at boot so we don’t have to wait 10 minutes for the first read.

  1. It looks like that it’s tied to the ADC sensor. Wouldn’t the ADC sensor still function fine without - component.update: battery_level ? Or does this just force it to happen on boot?

You’re right, just forcing an update.

Also, have you tested how big the detection area is? Wondering if you can use this to pickup a gocart in a driveway or a bicycle?

In my testing I used a set of nail clippers to trigger it. A vehicle typically makes more drastic changes in the reading but a go-cart should work, bike may too…

Thanks for the questions, I hope this is helpful!

How sensitive is the magnetometer?
I had the foresight to install some empty PVC under my driveway when it was being replaced about 5-years ago. I had no clue what I was going to use it for, but this seems like a good project.

I have already figured that I can put an ESP8266-01 in the car that is always trying to connect to the WiFi, and for Home Assistant to occasionally ping for it. This way I will know if the car is in the vicinity, but your project would tell me if it’s in the driveway.

Even though mine is in a junction box inside a 4" pipe under some gravel it’s still sensitive enough to detect a vehicle. Hard to say if it would work under a driveway but it might…

Has your detector been through a thunderstorm yet? Loop detectors and magnetometers are prone to false detects from lightning strikes. Just wondering how your’s is working?

Great project. I would be a bit concerned about that battery life if there is no external power involved. Constantly running WiFi could be an issue.
BTW did you know that you have to modify the WEMOS a to enable external antenna? https://raspi.tv/2017/how-to-use-external-antenna-on-wemos-d1-mini-pro-surface-mount-rework-video
Just saying that because I found out just recently and it make quite a difference :wink:

Hey, thanks for sharing!I didn’t know one could be added… cool!

As for external power: there is a solar panel trickle charger that tops off the battery all day. Haven’t had any issues so far, that is why I wanted the battery monitor in place though.

Yea, without that resistor being switched the external antenna just bring more noise to the on board antenna and makes link quality worse.

Oh. I misunderstood your reply. Yes I did know you have to move the resister! Interestingly enough on some of the boards I’ve received it’s already been moved. This has introduced problems when I thought I was using the inbuilt antenna! Always check!

It’s a zero-Ohm resistor. Why couldn’t you just use a solder bridge?

You project looks great. Just a question how accurate is the compas QMC5883L ? Actually my readings seem completly strange. It does need a calibration ? How far from the ESP is your sensor to get accurate values ?

Thank you! I didn’t bother with any calibration or anything since my project doesn’t really depend on knowing a correct compass value. The QMC is ~20ft from my ESP.

Ok because my first attempts gives completly crazy value… Unusable sadly :frowning:

Very neat.

I was using a Cartell CP-2 wand that I got on ebay for $75 10 years ago. It was used to detect vehicle motion in my driveway and it worked flawlessly for 10 years. Unfortunately, 3 weeks ago, lightning struck the ground near where the CP-2 wire was run and damaged it along with several other pieces of equipment. While I couldn’t find a deal like I did 10 years ago, I got a new Cartell CP-4 for $160. I thought about picking up the other type of wands that require external circuitry to operate to see if I could get them working with an esp8266 but they were still $75.

What I would really like to be able to do is detect the direction the vehicle is traveling. While I can do this with two Cartell CP-4 sensors, I don’t want to spend another $160 or dig 200 more feet of trench. Cartell has a wireless model (CW-SYS) but I can’t find pricing for it or anyone selling it other than installers and dealers.

Can your creation detect direction on its own and do you think it would be sensitive enough to be placed along side a 12’ wide driveway and still detect vehicle motion on the far 6’ side? Since I’m only detecting driveway motion and not opening gates or anything else, occasional false alarms are not a concern.

This may not be ideal but just throwing it out there: How about a cheap camera? You can then use OpenCV to do the detection. I don’t know your skills set so feel free to ignore.

I don’t think you can use a single magnetic loop to detect direction but it would definitely be possible to do it with two.

It’s been a while since I used OpenCV. I was using it to measure vehicle speed back when we had a problem with kids racing through the neighborhood. The sheriff’s office was of no help until I sent them the video along with license plates of the offenders.

With OpenCV. I was never able to get accurate enough results with driveway motion, especially at night. Way too many false detections during the day and it detected bugs in the IR as headlights at night and nothing when headlights were off. I didn’t have the patience to spend more time on it.

1 Like

This is a great project. I really appreciate you sharing it. I still don’t understand why is that you need the extra voltage divider besides the one that the D1 mini already has. Sorry about my ignorance. Thanks again for the project