I spent quite a while trying different ideas to get a working & fully local license plate setup and I am quite happy with this one so I am sharing a guide on how to do this.
Prerequisites
- You must have CodeProject.AI setup with the license plate plugin installed, that is outside the scope of this guide.
- Frigate+ is recommended for its built in license plate detection, however this can definitely work without Frigate+ with some modifications that are called out in the comments
Preliminary Info
The approach in this guide requires using a python script to run some of the logic because it would be much more difficult to do in a Home Assistant automation. This script is called directly from home assistant.
Python Script
- Create a folder to hold your scripts (if you don’t already have one) inside of the Home Assistant
/configdirectory. For this guide I will call mine/config/bashscript - Create a file called
license_plate.pyin/config/bashscript
The contents of the file should be
import datetime
import io
import json
import requests
import sys
from PIL import Image
# host for frigate instance
FRIGATE_HOST = "http://10.0.0.0:5000"
# host for CodeProject.AI instance
CODE_PROJECT_HOST = "http://10.0.0.0:4100"
event_id = sys.argv[1]
json = requests.get(f"{FRIGATE_HOST}/api/events/{event_id}").json()
camera = json["camera"]
# get clean snapshot if event has bended otherwise get snapshot without bounding boxes
if json["end_time"]:
img_bytes = requests.get(f"{FRIGATE_HOST}/clips/{camera}-{event_id}-clean.png").content
else:
img_bytes = requests.get(f"{FRIGATE_HOST}/api/events/{event_id}/snapshot.jpg?bbox=0").content
img = Image.open(io.BytesIO(img_bytes))
attributes = json["data"]["attributes"]
# Checks to ensure that license plate is detected
# this will need to be removed if not using a model that detects license plates (like frigate+)
if not attributes:
with open("/config/bashscript/detected_plates.log", "a") as log:
log.write(f"[{datetime.datetime.now()}]: {camera} - could not find license plate in {json}\n")
exit(0)
box = attributes[0]["box"]
detect = requests.get(f"{FRIGATE_HOST}/api/config").json()["cameras"][camera]["detect"]
known_res = (detect["width"], detect["height"])
# find coordinates of image to send to detector
x_mid = (box[0] + (box[2] / 2)) * known_res[0]
y_max = (box[1] + box[3]) * known_res[1]
cropped_image = img.crop((max(0, x_mid - 150), 0, min(known_res[0], x_mid + 150), known_res[1]))
cropped_image.save("/config/bashscript/crop.jpg")
# you will likely need to create multiple iterations for each vehicle
# 0/8/B for example are often mixed up
known_plates = {
# Bob's Car
"ABC128": "Bob's Car",
"ABC12B": "Bob's Car",
# Steve's Truck
"123TR0": "Steve's Truck",
"123TRO": "Steve's Truck",
}
with open('/config/bashscript/crop.jpg', 'rb') as fp:
response = requests.post(
f'{CODE_PROJECT_HOST}/v1/image/alpr',
files=dict(upload=fp),
)
print(response.json())
plates = response.json()
plate = None
if len(plates["predictions"]) > 0 and plates["predictions"][0].get("plate"):
plate = str(plates["predictions"][0]["plate"]).replace(" ", "")
score = plates["predictions"][0]["confidence"]
print(f"Checking plate: {plate} in {known_plates.keys()}")
with open("/config/bashscript/detected_plates.log", "a") as log:
log.write(f"[{datetime.datetime.now()}]: {camera} - detected {plate} as {known_plates.get(plate)} with a score of {score}\n")
if plate in known_plates.keys():
print(f"{camera} - Found a known plate: {known_plates[plate]}")
else:
plate = None
else:
with open("/config/bashscript/detected_plates.log", "a") as log:
log.write(f"[{datetime.datetime.now()}]: {camera} - No plates detected in run: {plates}\n")
if plate is None:
print(f"No valid results found: {plates['predictions']}")
sys.exit()
vehicle_name = known_plates[plate]
# Add Sub Label To Car Event
requests.post(f"{FRIGATE_HOST}/api/events/{event_id}/sub_label", json={"subLabel": vehicle_name, "subLabelScore": round(score, 2)})
There are a few things to note here:
- CodeProject.AI port might be different depending on where and how it is setup
- Some characters are commonly mixed up like 0/O and 8/B, these variations should be added so this works more reliably.
- This will create a file
detected_plates.logwhich will provide info on what plates are detected and if it matches a known plate - The latest image sent to ALPR will be saved as
/config/bashscript/crop.jpg
Home Assistant Configuration
The next step is adding this python script to the Home Assistant configuration.yaml file. This is done via the shell_command type:
shell_command:
# run license plate inference for vehicle
check_license_plate: python3 /config/bashscript/license_plate.py "{{ event_id }}"
Home Assistant Automation
The final step is setting up the Home Assistant automation, this will run on a car event only when it has a license plate attribute detected. It will then run 3 iterations of checking for the license plate and stop when a license plate is detected.
This single automation will run on all cameras that detect a car with a license plate in the zones defined, so if you have a driveway zone specified for the garage and driveway camera it will run on both.
alias: >-
Outside - Driveway - Frigate Try To Read License Plate - License Plate
Seen
description: ""
trigger:
- platform: mqtt
topic: frigate/events
id: frigate-event
payload: license_plate
# will just want to check that the label == "car" if not using a model that supports license_plate
value_template: "{{ value_json[\"after\"][\"current_attributes\"][0]['label'] }}"
variables:
after_zones: "{{ trigger.payload_json[\"after\"][\"entered_zones\"] }}"
before_zones: "{{ trigger.payload_json[\"before\"][\"entered_zones\"] }}"
current_zones: "{{ trigger.payload_json[\"after\"][\"current_zones\"] }}"
camera: "{{ trigger.payload_json[\"after\"][\"camera\"] }}"
id: "{{ trigger.payload_json[\"after\"][\"id\"] }}"
label: "{{ trigger.payload_json[\"after\"][\"label\"] }}"
condition:
- condition: and
conditions:
# setup zones that you want to look for cars with license plates for detection
- condition: template
value_template: >-
{{ ["street", "driveway"] | select("in", after_zones) | list | length
> 0 }}
alias: Has been in the street
- condition: template
value_template: "{{ [\"driveway\"] | select(\"in\", current_zones) | list | length > 0 }}"
alias: Was in the street and is now in the driveway
- condition: template
value_template: "{{ trigger.payload_json['after']['sub_label'] is none }}"
action:
- choose:
- conditions:
- condition: trigger
id: frigate-event
sequence:
- service: shell_command.check_license_plate
data_template:
event_id: "{{id}}"
camera: "{{camera}}"
- repeat:
count: 2
sequence:
- wait_for_trigger:
- platform: mqtt
topic: frigate/events
payload: "{{ trigger.payload_json['after']['id'] }}"
value_template: "{{ value_json['after']['id'] }}"
timeout:
hours: 0
minutes: 10
seconds: 0
milliseconds: 0
continue_on_timeout: false
- condition: template
value_template: >-
{{ wait.trigger.payload_json['after']['sub_label'] is none
}}
- service: shell_command.check_license_plate
data_template:
event_id: "{{id}}"
camera: "{{camera}}"
default: []
trace:
stored_traces: 20
mode: parallel
max: 10