FaceID — trainable face recognition for Frigate with a proper review UI (Home Assistant app + standalone)

Would it be possible to add a bulk-delete option for ignored people, such as “Select all” and “Delete selected”?

An automatic cleanup option would also be helpful—for example, deleting ignored people after a configurable number of days.

@lemuba

Both are in v0.7.1 — one exactly as you asked, the other deliberately a bit different.

Bulk selection is there now. The Ignored tab has select all, clear and delete
selected
at the top, spanning every group at once. Each group already had its own delete
button, but with a dozen groups that was pointless clicking. The confirmation spells out
what it means: those people can show up in the review queue again afterwards.

On automatic cleanup after N days, I went a different route, and I think the reason
matters more than the feature.

An ignore anchor isn’t a temporary note — it is the memory of who you don’t want to hear
about. Delete it after 30 days and the delivery driver is back in your review queue, then
gets ignored again, then comes back. Age is simply not the right criterion: an anchor from
three months ago describes that person exactly as well as yesterday’s.

What I suspect is actually bothering you is something else. With ignore_learning on,
every unambiguous ignore match adds another anchor, and that grows without bound —
especially if a street or pavement is in frame. The list fills up not because anchors get
old, but because there are too many of them.

So v0.7.1 adds max_ignore_anchors in Settings (0 = unlimited, so nothing changes
unless you set it). When a group exceeds the cap, FaceID drops the most redundant
auto-learned anchor — the one most similar to the others, so unusual angles survive. It
never removes an anchor you added by hand. Same principle as the per-person photo cap.

Try something like 15 and see whether the list stays manageable. If your case really is
“these ignores were a mistake and I want them gone wholesale”, that is what select-all is
for — one click, one confirmation.

If you do hit a situation where age genuinely is the right criterion, say so with the
concrete case. I would rather be argued out of this than have guessed wrong.

Thanks for implementing the batch delete feature – it’s exactly what I was hoping for. Much appreciated! I’ll also be happy to test the other new features you’ve added.

On a different topic:

I’m running FaceID as a Home Assistant app, so I don’t have direct access to the FaceID filesystem or the backup files themselves.

While testing the Backup & Restore functionality via the Home Assistant Companion app on my iPad, I noticed a couple of inconsistencies:

  • The Download Backup button appears to always download only the most recently created backup. Is that intentional? I was expecting some way to choose an older backup if multiple backups are available.
  • Also, tapping Download Backup doesn’t actually download a backup file on iPad. Instead, it opens the backup directly in the browser/viewer as plain binary data (see screenshot below), resulting in a page full of unreadable characters instead of prompting me to save the backup as a file.

I’m not sure whether this is a limitation of Home Assistant’s file handling or something FaceID could work around, but I thought I’d mention it since it makes restoring or archiving backups a bit difficult when using FaceID through Home Assistant.


Please make the 48px requirement a setting we can change easily in the Home Assistant “App” version of this. Most of my detections come right under the threshold and I’d like to see just how bad it is with a value less than 48px.. :slight_smile:

@lemuba Good catches, both of them — and one was worse than you described.

Saved backups are now listed and individually downloadable (Settings, under the
automatic backup section): date, size, and a download link per file.

Your observation was right, my mechanism was slightly different from what you assumed:
the button never downloaded “the latest backup” — it built a fresh archive on every
click. The files written by backup now and by the daily job were not reachable from the
UI at all. On a normal install you could still fetch them over SSH; running as an app,
they were effectively invisible. That was an oversight, not a design decision.

The iPad problem should be fixed, but I cannot verify it myself. The
Content-Disposition: attachment header was already being sent — Safari simply ignores
it for application/gzip and renders the archive inline, which is the character soup you
saw. The download now uses application/octet-stream plus a download attribute on the
link, which normally persuades iOS to save the file.

I have no iPad here, so please tell me whether it actually behaves now. If it still opens
inline, the next thing to try is a long-press on the link and “Download Linked File” —
that would tell us whether it is the header or Home Assistant’s ingress rewriting the
response.


@nrgz28 You were right, and it was worse than a missing option: the app wrote a fixed
min_face_px: 48 into its configuration regardless of what anyone set. There was no way
to change it from the app at all.

v0.7.2 makes it an app option and live-editable under Settings, together with the two
other knobs that matter for the same problem:

  • min_face_px — the limit in the log line “largest face NNpx < min_face_px”
  • max_attempts (default 6) — FaceID retries while an event is still running, so more
    attempts mean more chances to catch the moment the face is largest. Often this helps
    more than lowering the size limit, and it costs nothing in accuracy.
  • det_size (default 640) — detector input size; higher finds smaller and more
    distant faces at the cost of CPU.

By all means try 32 and see what happens — that is exactly why it should have been
adjustable. What to expect, honestly: below roughly 40 px, ArcFace embeddings get
noticeably unreliable. You will get more matches, but also weaker ones, and eventually
strangers start scoring high enough to look like family.

Watch it in the LOG tab while you tune: every match is logged with its score, and
every miss says how large the face actually was. If matches start landing barely above
your threshold, or a stranger turns up with a name, you have gone too low — put it back
up a notch. Lowering the limit buys attempts, not accuracy.

(There are measurement scripts in the repo for exactly this question, but they need shell
access to the container, so they are no use inside the app. Making that analysis available
in the UI is on my list — if it would help you, say so and I will move it up.)

Thank you for reacting so quickly! I’ve been using FaceID for a couple of days now and for me, it seems to be better at detecting and managing faces than Frigate’s own mechanism. As far as the analysis tools… I dont know what I’m missing, since I can’t really run them. Heh

Maybe for your next iteration. adding a cool icon in the App area in HA would be icing on the cake, instead of what looks like the default icon.

Many thanks again for your quick response! I can confirm that everything is working perfectly now. All backup downloads are working flawlessly on my iPad with the Companion App.

Two things shipped that came directly out of this thread.

v0.8.0 — the calibration analysis now runs from the UI. Settings → Does it actually
work?
, and it needs no shell, so it works inside the Home Assistant app.

@nrgz28 this is the “I can’t really run them” from your post. Fair point, and it was a
real gap: the project tells everyone to measure instead of guess, then put the measuring
behind a shell that app users don’t have.

It runs two things in the background, both without any manual labelling:

  • Leave-one-out over your gallery. Every reference photo is matched against the
    gallery it was removed from. Real ground truth, no work for you.
  • A probe over recent events (1–7 days, your choice): how many recognitions happened
    and how much headroom each one has above your threshold.

The result leads with the number that actually decides things: how high a stranger got.
Ignore anchors and cross-person matches both count towards it. On my own setup strangers
peak around 0.19–0.34 against a threshold of 0.45 — that gap is what tells me the
threshold can come down. If the run finds even one misassignment, it says so and tells you
to raise the threshold instead. It will not suggest lowering anything on a gallery that is
already confusing people.

Two details that matter for the numbers being honest: events whose face is already in your
gallery are excluded from the probe (they score ~1.0 and would flatter a low top_k), and
the same code now backs both the UI and the scripts, so they cannot drift apart.

If you lowered min_face_px after the last release, this is the way to check what it
actually cost you.

v0.8.1 — the app has a proper icon. @nrgz28 also asked for that.

@lemuba thanks for confirming the iPad backup fix — that was the one thing I couldn’t test
myself, and knowing it works closes it properly.

v0.9.0 — the biggest recognition gain so far came from an assumption I never checked.

Short version: if FaceID recognises far fewer people than you expected, it is probably not
your gallery or your threshold. It is the snapshot.

How I found it. I had spent weeks on the gallery — more reference photos, a calibrated
threshold, match_top_k. A scheduled re-measurement two weeks on showed the honest
result: nothing had improved. 135 → 148 photos, leave-one-out identical at 88 correct,
mean score 0.517 → 0.519.

The real finding was in a line I had been ignoring: of 39 person events over seven days,
only 6 had a usable face at all. Whether the gallery holds 135 or 148 photos cannot
matter when five out of six events never get that far.

So I counted the reasons instead of guessing them (42 events):

what the snapshot gave share
no face detected at all 55%
face below min_face_px (median 34px) 14%
detection too uncertain 12%
usable 19%

The cause: Frigate picks its snapshot by highest person score. That is not the same
criterion as “a face is visible” — quite often the clearest view of a person is the
moment they turn away from the camera. The clip usually does contain a good frame. Of
twelve failed events re-checked frame by frame, nine had a perfectly good face
(det 0.68–0.87).

Three theories I had, all measured, all wrong:

  • “It’s the IR/night shots.” IR failed 11 times, colour 12. No difference.
  • “People are too far away.” Not one crop was narrower than 120px.
  • “4K frames get downscaled to det_size, so faces vanish.” This one felt certain.
    det_size 1280 instead of 640 produced identical results at twice the cost
    (11.1s vs 5.4s per event).

What was actually left is the viewing angle — which is why there is no single number
in the docs. Per camera:

camera rescued from the recording
front door, at head height 4/4
child’s room, mounted high 0/4
garden, zoomed 0/2

Where people walk towards the lens, nearly every failed snapshot is recoverable. Where the
camera looks down at them, the clip holds no face either — scanning cannot invent one.

The change: Settings → Search the recording (clip_fallback, default on). When an
event ends and the snapshot never produced a face, FaceID samples 12 frames across the
clip and takes the best. It runs only for events that produced nothing anyway, on its own
thread with a short queue, so live recognition and presence updates are never delayed and
a burst of events skips scans rather than building a backlog. Turn it off if your hardware
is tight.

One deliberate detail: the frame is chosen by detection quality, never by gallery
similarity
. Picking whichever of twelve frames looks most like someone you already know
would improve the numbers and quietly increase misassignments. Identity is decided
afterwards, exactly as on the snapshot path.

What it did here: six front-door events that were previously lost completely — not
recognised, not even queued for review — produced two direct recognitions and four
candidates with the correct name suggested. As a bonus, faces from clips came out
158–258px wide against a snapshot median of 82px, so they make better references too.

If you want to check your own setup before changing anything: Settings → Does it actually
work?
now reports how many recent events yielded a face at all. If that number is low,
this release is likely to help you — and if your cameras look down on people, it honestly
may not.

This latest version is even better at recognizing people than Frigate itself. Great job! And thank you for exposing all the fine tuning settings in the Settings tab!

One issue I ran into.. for your future builds… I can’t easily rename a person. I misspelled someone’s name when adding a person and now I can’t find a way to make it right.

Thanks — and you found a real gap. Renaming a person genuinely was not possible: the only
route was deleting them and starting over, which threw away every reference photo and every
assignment. A steep price for a typo.

Fixed in v0.13.0 — click the name on the person card in the Persons tab.

Only the display name changes; photos, embeddings and assignments stay exactly as they are.
The internal folder keeps its original slug on purpose — it is referenced by stored files
and image URLs, and a misspelling is no reason to migrate data. What you actually see is
the name, and that is what changes: in the UI, in the presence sensor, and in the Frigate
sub_label from the next recognition onwards.

Two guards: renaming to a name someone else already has is refused with that reason, and so
is an empty name.

One thing it does not do, to be clear: Frigate events that were already tagged keep the
old sub_label. Rewriting history there would mean walking back over every past event, and
I would rather not have a rename touch data that far away. New events get the corrected
name.

On the fine-tuning settings — glad they earn their place. Worth knowing that Settings →
Does it actually work? now also tells you whether a change helped, without needing shell
access: it runs a leave-one-out test over your gallery and reports how high a stranger
got, which is the number that decides how far the threshold can safely come down.

Any tips on reducing the amount of RAM FaceID needs ? It seems to be the most hungry addon in my homeassistant! This is from the “Glances” addon… Using 1.14 out of my 16G of RAM :slight_smile:

app_410739ac_faceid running 2 hours 376 1.14G 16.0G 0 0 1.77Mb 10Kb

Short answer: 1.14 GB is normal, and there is unfortunately no setting that meaningfully
reduces it. I went and measured rather than guessing, because I had a few theories and all
of them turned out wrong.

Where the memory actually goes

Python + numpy + opencv + onnxruntime 64 MB
loading the buffalo_l models +572 MB
first recognition +0 MB

Inside that, the two models split as:

det_10g.onnx — finding faces (SCRFD) +51 MB
w600k_r50.onnx — face → 512 numbers (ArcFace) +333 MB

w600k_r50 is a ResNet-50: roughly 25 million parameters, unpacked into memory as float32
tensors plus the intermediate buffers onnxruntime keeps for the forward pass. That is the
bill, and it is the same one Immich and CompreFace pay for the same models.

FaceID already loads only what it needs (allowed_modules=["detection", "recognition"]).
The pack also contains 3D landmarks, 106-point landmarks and gender/age estimation —
another ~140 MB that never gets touched.

Three things that sound like they should help, and do not

All measured on the same machine, same gallery:

  • OMP_NUM_THREADS at 1, 2, 4 or default — no change (632–645 MB throughout)
  • det_size at 320, 480, 640 — no change
  • disabling onnxruntime’s CPU memory arena (enable_cpu_mem_arena = False) — no change,
    verified twice

I was fairly confident about the arena one in particular. It made no difference at all.

What det_size does change is CPU, and quite a lot

det_size per image
640 (default) ~185 ms
480 ~107 ms
320 ~60 ms

Identical memory, three times the speed. If FaceID is bothering you on CPU rather than RAM,
that is the lever — but lower values find small and distant faces less reliably, so check
what it costs you with scripts/why-no-face.py instead of going by feel.

If you genuinely need less RAM

The only real reduction is a smaller model family — buffalo_s uses MobileFaceNet instead
of ResNet-50, roughly a third of the memory. It recognises less well, particularly on small,
angled or IR faces, which are exactly the hard cases anyway. And embeddings from the two
model families are not compatible, so switching means re-enrolling every reference photo.
On a machine with 16 GB I would not make that trade.

All of this is now in the README under Requirements → What it actually uses, including
the measurements that came out negative — those seemed worth writing down precisely because
they are the things everyone tries first.

@skycryer, thanks a lot for sharing this - started using it yesterday and so far looks very promising.

Perhaps I am overseeing something but is there a way to honor the zones of Frigate in what is being included in scanning and what is not ? E.g. I have some zones from a camera where I’d like FaceID to scan and others where it shouldn’t.

Thanks!

Good question, and the honest answer is: FaceID itself has no notion of zones. The only
filter it has is cameras, which is all-or-nothing per camera.

But you can already have exactly what you are asking for, without waiting for me — and I
would try that route first, because the obvious implementation inside FaceID turns out to
be a trap. More on that below.

The way that works today: let Frigate decide

FaceID only ever touches an event once Frigate reports has_snapshot. No snapshot, no
work — the event is ignored entirely, including the clip and live-frame fallbacks. So
Frigate’s own zone gating does the filtering for you:

cameras:
  your_camera:
    snapshots:
      required_zones:
        - zone_you_care_about

Now only people who actually entered that zone produce a snapshot, and only those events
reach FaceID. You get Frigate’s full zone semantics for free — inertia, loitering_time,
the object anchor rules — instead of a cruder copy of them inside my app.

One side effect worth knowing: those events then have no snapshot in the Frigate UI either.
If you rely on snapshots for other things, that trade may not be worth it for you.

Why I would not just add a zone filter to FaceID, and what I measured

The obvious feature is “only scan events whose zones include X”. I checked what that
would have done to my own recognitions before writing it, and the result is uncomfortable.

Over seven days, on my one camera that actually has a zone defined:

successful recognitions on that camera 28
event carried the zone 16
event carried no zone at all 12 (43 %)

Those 12 are people the system recognised correctly — standing at my front door, inside the
area the zone covers. Frigate simply never registered the object in the zone, because being
visible in a region and being counted as in a zone are different things: the zone test
uses the object’s anchor point and has its own timing rules. A naive filter would have
thrown away nearly half of my working recognitions.

And that number is a lower bound, not the real cost. I queried the finished events,
where zones is complete. FaceID works on events while they are still running, and early
on entered_zones is usually still empty — the person has not reached the zone yet at the
moment of the first snapshot, which is often the snapshot that works. A live filter would
drop more than 43 %, not less.

This is measurable on your own setup before you commit to anything. Pull a handful of
events you know FaceID got right and look at whether zones is populated:

curl -s "http://<frigate>:5000/api/events?label=person&limit=40" | jq '.[] | {camera, zones}'

If your zones cover the approach path generously and come back populated consistently,
gating is safe for you. If they look like mine, required_zones on snapshots is the better
lever, because it fails in the same direction as your existing snapshot config rather than
silently costing you recognitions.

What would actually convince me to build it

If required_zones does not fit — say you want snapshots kept for everything but FaceID to
skip the street-facing part of one camera — tell me which of the two you are after, because
they need different implementations:

  1. Save CPU — skip events outside the zone entirely. Cheap, and the 43 % problem hits
    full force.
  2. Avoid wrong presence reports — still scan, but do not publish a person seen only
    outside the zone. Costs nothing in recognitions, and it is the version I would actually
    want to write.

Which one is your case?

Thanks for the swift answer, @skycryer ..

I am after the scenario you refer to as “ Avoid wrong presence reports”. Basically it’s all about filtering what triggers notifications on my phone. I don’t really care about people in - for example - a street zone - nor known faces in a driveway zone. Unknown faces in the same zone would be good to be able to trigger on. I am currently calling Frogate’s API to retrieve the event details but see your point regarding when zones are being populate by Frigate.

That is the version I wanted to hear, so I built it — v0.16.0, out now.

Not as a filter, though. faceid/event now carries the zones instead:

{"person": "unknown", "score": 0.31, "camera": "driveway",
 "event_id": "1786945313.350301-xiec8d", "ts": 1786945318.4,
 "zones": ["driveway_zone"]}

The same payload also appears as the last attribute on the presence sensor, and unknown
faces are published on that same topic as person: unknown — so both halves of your case
run off one trigger. You can drop the Frigate API call.

Why data rather than a filter, given you asked for the safe variant anyway

Your rule is known people silent in the driveway, unknown people alerting in the same
zone, nothing at all from the street
. That is per-person and per-zone at once. I cannot
express it as a config option without inventing a small rules language, and you can express
it in four lines of automation. So the useful thing for me to ship is the zone, not a
decision made on your behalf.

An example is now in
docs/ha-automation-example.yaml:

- alias: 'FaceID: stranger in the driveway'
  mode: queued
  triggers:
  - trigger: mqtt
    topic: faceid/event
    value_template: '{{ value_json.person }}'
    payload: unknown
  conditions:
  - condition: template
    value_template: "{{ 'driveway' in (trigger.payload_json.zones | default([])) }}"

Known people need no rule at all here — they simply never match payload: unknown.

The one thing to get right

An empty zones is not “outside every zone”. It looks exactly the same when the camera
has no zones and when Frigate never placed the person in one — and on my zoned camera that
was 12 of 28 correct recognitions. So write your conditions as zone is present rather
than zone is absent. The example above does that deliberately: an event with no zone
information stays quiet rather than alerting. If you would rather have a false alarm than
miss a stranger, invert it — but make that an explicit choice, because the two readings
differ far more often than the wording suggests.

That is also why I would still reach for snapshots: required_zones: in Frigate if you
ever want hard gating on a camera. It fails in the same direction as your snapshot config,
and FaceID never sees those events at all.

Worth saying plainly: this went in because you answered the “which of the two” question
instead of just asking for the feature. The filter version was the one I was ready to
build, and it would have quietly cost you recognitions.

Thanks a lot, @skycryer! Will give it a spin when I return to home :grinning_face: