Object detection for scaring foxes and counting eggs

jenniferb · February 27, 2024, 7:02pm

Hi,

I just built an HA yellow and am proud of myself. I now have lights in the house turning on and off, which is great.

My project - I have backyard ducks and would like to create a duck surveillance system to:

a) keep metrics on which duck laid eggs (the ducks all have distinct markings, and I have a large training set of photos).
b) count the ducks to know what time they all return to the coop at relative to sunset
c) watch for the fox that visits and turn on a sprinkler if detected

I was inspired to start this project by this thread: Object detection: did my chickens make an egg? (no longer need to look myself) - #22 by mantaalex

I think I can figure the software and training part out, but I’m starting from scratch and could use some tips on what hardware to buy. What cameras should I get? I think for inside the coop I just need night vision and still periodic photos. I would detect the egg, then go back a few minutes and detect the duck near that area to get a metric. For the fox detection I would need something more dynamic. I see there are cameras with animal detection systems on them already. Should I bother with that or just do my own? What cameras would you / have you used for these types of situations?

I have an extra raspberry pi 4 B which I bought by accident. Should I set this up as an NVR running Frigate? Should I get a coral USB?

Thanks in advance for any tips. Looking forward to learning a lot.

ddaniel · February 27, 2024, 7:20pm

I like this. I have foxes in my yard too. I didn’t know that till I saw them. Wild boars and deer. And that is incredible because I live in the city.
But never the less if you plan to use cameras you will probably need something stronger than a pi.
I will suggest to you, as I done to all others, buy a used ddr3 ram comp because you can get it cheap, memory for it is cheap and you can find really great gpu, nvidia, for cheap. Try to find something that has preferably intel cpu 2nd gen or later with avx, avx2 support. I have google coral tpu usb but I think that this works the best in combination with nvidia gpu.
Frigate as per se is nvr so everything will be stored on your local ssd.
For cameras… well I think that the best option will be poe camera. Yes, you will need a poe switch and run cables.
You will probably have to go for onvif camera.

Edwin_D · February 27, 2024, 7:25pm

I guess this Tesla robot would scare a fox too

CO_4X4 · February 27, 2024, 7:51pm

I saw that first article when it was first written about the automation to detect eggs and thought it was brilliant, I’ll anxiously await how your project progresses as these principals could play into a system I’ve wanted to build for years: raccoon deterrent.

jenniferb · March 26, 2024, 8:14pm

For this project I ended up getting the following:

a Raspberry Pi 5 with a Coral TPU
a Reolink 510-WA

Not knowing much about NVIDIA systems, I went with the Pi/Coral due to this tutorial, which seemed achievable:

The Reolink camera was easy to integrate with HA and came with animal detection out of the box, which actually worked. Unfortunately the fox stopped visiting a few weeks ago, so I have not pursued this track. I wanted to use the Coral, so I decided to try to train it to recognize the ducks individually.

We had two Blink cameras already which we installed in the duck house. I tried to use the HA integration to take pics, but ran into an issue due to our lossy network, which was promptly fixed by the pyblink team. It was unclear when the fix would make it into HA, so I set up a cron script to use pyblink directly to take pics inside the duck house overnight, and used the HACS Media Card to display the pictures in HA.

I set up a local copy of LabelStudio to tag the pictures of the ducks.
I used google’s tflite modelmaker on my laptop to generate a model to use for detecting my ducks following this tutorial:

https://www.youtube.com/watch?v=-ZyFYniGUsw

I then set up some cron scripts to upload new pictures to LabelStudio in the morning (using the existing model and the Label Studio SDK to predict the labels) and re-train the model in the evening (with the idea that I spend some time labeling during the day). The accuracy gradually improved.

As this was all going on, my ducks started a new behavior where they created a shared nest and buried their eggs in it. I was able to review the overnight pics on HA to try to guess where the eggs might be buried. We would rarely catch anyone in action, however there was a common pattern of the ducks sitting in the same spot over time. Since this has always been a machine learning project looking for a problem, I decided to use my object detection model to find a spot which had been occupied by the most ducks overnight, and show that result in HA.

Which saves time on the egg-finding.

This has been a fun project. That said, I would warn anyone looking to do something similar that the fact that the coral toolset and simplified tflite model maker are quite stale made development dependent on some older docker images and not portable to, for instance, google collab. This is a big red flag for the coral project. Ultimately, the fast inference times of the coral chip were not necessary as the object detection is not done in real time. I was disappointed that Frigate does not support custom coral tpu models out of the box, but if I wanted to integrate real time object detection in video streams I am confident I could do it with some coding in OpenCV, using MQTT to have the results show up in HA.