License Plate Recognition using Gemini 3.5 Flash API (Gate Opener & Notifications)

Hello everyone!

I want to share a project I created to detect vehicle license plates using a Home Assistant camera through the Gemini 3.5 Flash API. It is extremely fast, highly intelligent (reads plates perfectly even at difficult angles), and does not require heavy local add-ons. When a recognized car arrives, it automatically triggers the gate switch and sends personalized notifications (TTS via Alexa and a pop-up alert on an LG webOS TV).

Since this setup requires a small helper script, installation is done in 3 simple steps:

1. The Python Script

Create a file at /config/www/tmp/analisar_matricula.py and paste the following code (don't forget to insert your own Gemini API Key in the configurations section):

Python

import sys
import requests
import base64

# ==============================================================================
# CONFIGURATIONS (The user must change these details)
# ==============================================================================
API_KEY = "PASTE_YOUR_GEMINI_API_KEY_HERE"
MODELO = "gemini-3.5-flash"
IMAGE_PATH = "/config/www/tmp/check_plate.jpg"
# ==============================================================================

def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode('utf-8')

def analise_matricula():
    try:
        base64_image = encode_image(IMAGE_PATH)
    except Exception as e:
        # If the image doesn't exist at the moment, warn and exit safely
        print(f"Error: Image not found at {IMAGE_PATH}")
        sys.exit(1)

    url = f"https://generativelanguage.googleapis.com/v1beta/models/{MODELO}:generateContent?key={API_KEY}"

    headers = {
        "Content-Type": "application/json"
    }

    payload = {
        "contents": [{
            "parts": [
                {
                    "text": "Read the license plate in this image. Respond ONLY with the license plate text (e.g., AA00BB). If there is no license plate, respond 'NADA'."
                },
                {
                    "inline_data": {
                        "mime_type": "image/jpeg",
                        "data": base64_image
                    }
                }
            ]
        }]
    }

    try:
        response = requests.post(url, headers=headers, json=payload, timeout=8)
        
        if response.status_code == 200:
            resultado = response.json()
            # Safe extraction of the license plate text returned by Gemini
            texto = resultado['candidates'][0]['content']['parts'][0]['text']
            print(texto.strip())
        else:
            # Show the error if the Gemini API fails
            print(f"API Error: {response.status_code}")
            
    except Exception as e:
        print(f"Connection error")

if __name__ == "__main__":
    analise_matricula()

2. The Shell Command

Add the following lines to your configuration.yaml file, then go to Developer Tools -> YAML and Reload Shell Commands (or restart Home Assistant):

YAML

# Command that executes the license plate reading Python script
shell_command:
  analisar_matricula: "python3 /config/www/tmp/analisar_matricula.py"

3. The Blueprint

You can import the Blueprint directly by copying the URL below and adding it to your Blueprints dashboard:

  • Import Link: https://gist.githubusercontent.com/Rveiga86/cbe638cbfa9f7987af72f59b1aedb3ad/raw/d05766668dbe4c74d45fe3850fa465870f1dcb1e/gemini_matriculas.yaml

Once imported, simply create a new automation from this Blueprint and fill in the required fields (Sensors, Gate Switch, License Plates, and Car Names) directly through the Visual User Interface (UI). No further YAML editing required!

I hope you find this useful. Let me know if you have any questions!

1 Like

This is awesome, there is so much room for improvement and expansion. So many ideas...

...or feature requests if you accept {?}:grin:

Of course I accept! This is my first project, I'm new to this and I want to improve, so any suggestions are welcome

Alright then!

Possible scenario:

Trigger: start the process of license plate recognition if a motion sensor/radar is triggered (there is a car in front of the gate)

Condition: open the gate if the plate is recognized by the first camera and belongs to one of the valid plates. The list of the plates is loaded from a CSV file located at Google Drive. The format is: Plate, Owner, Expire. "Expire" is a date (when plate is expired then do not open gate, send a message instead and turn on a red light)

Action: open the gate, turn on a green light and write this event in a calendar

There must be an alternative way to open the gate if the recognition fails. This can be done by another automation that reads NFC cards or listens to a remote control. Then, the second automation should trigger the recognition once again in a second camera and log the event to the calendar. Snapshots of the events, valid entry or not, are a welcome idea.

This is a draft...

Thanks for your feedback and for sharing this draft! You've brought up some really interesting concepts.

However, please keep in mind that this project is strictly designed for residential/private use (averaging 2 to 4 family cars per household), rather than a commercial or industrial parking lot. My main goal is to keep it simple, fast, and 100% local. Adding external dependencies like Google Drive CSV files or managing expiration dates would add too much complexity and create points of failure if the internet goes down.

That being said, I really like some of your suggestions for a domestic environment:

  • Alternative opening (NFC / Remote Control): This is an excellent Plan B for bad weather or when the camera fails to read the plate.

  • Status Lights: Turning on a green light via a switch (like a Shelly relay) when the plate is authorized is a great touch and very easy to implement.

I will definitely consider integrating these domestic improvements into a future update. Thanks again for the input!

I also like the idea of executing something on entry. The implementation should allow for a script to be called and that script can perform one or more actions.

"Yes, I took the light idea into account and expanded it into a full traffic light system using 3 lights! Here is how the logic works now:

  • Vehicle passes the photocell: The Red light turns ON (indicating the snapshot is being taken).

  • Processing phase: The Red light turns OFF and the Orange light starts flashing (indicating the Gemini API is analyzing the plate).

  • License plate approved: The Green light turns ON (and the gate opens).

  • License plate rejected: All 3 lights start flashing together as an alert.

I really liked how intuitive it became!"

1 Like