@blakeblackshear First off this is incredible! I’ve been using it for over a month now! Thank you!
What do you think about detecting a mail truck? So that you can be notified when the mail has been delivered. I don’t see a specific model for mail trucks in MobileNet though. I’m diving into this tonight.
It isn’t available. It is just a concept I thought of. I will create a github issue for it. Also, I could make that value something that could be set via MQTT as well.
All of those types of use cases will likely require training your own model. It is something I have thought about, but haven’t spent much time on yet.
Yeah I think that would be an ideal solution. I was trying to come up with a workaround that would save you time, but yeah I went back over the thread and saw some commentary regarding messages flooding mqtt haha
The imagenet dataset has categories for delivery van, articulated lorry, snowplow, etc… I think the general van category would work… You could try one of the imagenet trained coral models and see if they do what you need.
I bought a raspberry pi 4 to test this on, if it ever shows up, I’ll let you know how it works!
I’m seeing an issue where the best_person.jpg endpoint i.e. http://10.0.1.2:5000/garage/best_person.jpg is not updating to the latest best person in time for my HA automations to send the latest image. i.e. the person binary sensor runs -> camera snapshot of the best person camera/endpoint saved to a file -> pushover image alert. I sometimes get the old last_person from a few hours ago, not the latest image. I’ve tried adding a delay before the camera.snapshot in my automations, but this is not always working.
I believe this might be hardware related - i’m running everything on a Synology DS1819+ with Atom C3538 + 32GB ram. Frigate shows as taking roughly 35% of the CPU and about 1GB of RAM. There are ~10 other docker images on my Synology.
This project rocks btw - thanks for all the work on this, my Coral came this week and i’m amazed at how accurate / fast the camera alerts are now.
I thought frigate waited something like 2 seconds to decide between the stream of shots which one had the highest detection percentage before updating the best_person.
How often is your camera sensor configured to poll in homeassistant? If you poll every 2 seconds, and ensure you wait 4 seconds before saving the snapshot, I would expect it to work. It might be better if I ultimately push an image to homeassistant.
What would be even better would be an animated gif that goes back a second or so to capture some of the movement.
Wow that’s pretty cool, but I couldn’t get it to import. I named it frigate.txz and tried to import it via container->settings->import, but it said it was an invalid format… Am I missing something?
I’ll try to play around with the camera polling - right now it’s set to default. Do you push images with your setup? For now i’ll increase my automation delay time.
I thought the delay was on my end and i had an excuse to buy a nuc…
Okay, so I moved over to synology in the hopes that my issues were just related to VMware, but it looks like there’s something more than that going on here. I got it up and running with 1 camera at 1920x1080, 1fps and a second camera at 1080x1920, 1fps (landscape orientation). I’ve always run this way in the past because the detection works a lot better when the people are oriented properly in the video, and these cameras look down the side of the house so they are turned sideways to catch the whole frame.
What happens is that it only appears to draw bounding boxes on the portrait orientation feed… Then I went outside and walked around and when I went to the camera with landscape orientation, it stopped drawing bounding boxes around both… Now I have 2 feeds running with no bounding boxes… See the log below (from synology so it’s a little strange looking, I deleted some of the repetetive messages)
Is there some way to tell if it’s successfully communicating with the coral or not?
date stream content
2019-07-07 23:02:19 stdout queue full. moving on
2019-07-07 23:02:19 stderr Invalid UE golomb code
2019-07-07 23:02:19 stderr Invalid UE golomb code
2019-07-07 23:02:19 stderr KeyError: 'favicon.ico'
2019-07-07 23:02:19 stderr frame = cameras[camera_name].get_current_frame_with_objects()
2019-07-07 23:02:19 stderr File "/opt/frigate/detect_objects.py", line 79, in imagestream
2019-07-07 23:02:19 stderr for item in iterable:
2019-07-07 23:02:19 stderr File "/usr/local/lib/python3.5/dist-packages/werkzeug/wrappers/base_response.py", line 45, in _iter_encoded
2019-07-07 23:02:19 stderr return self._next()
2019-07-07 23:02:19 stderr File "/usr/local/lib/python3.5/dist-packages/werkzeug/wsgi.py", line 507, in __next__
2019-07-07 23:02:19 stderr for data in application_iter:
2019-07-07 23:02:19 stderr File "/usr/local/lib/python3.5/dist-packages/werkzeug/serving.py", line 292, in execute
2019-07-07 23:02:19 stderr execute(self.server.app)
2019-07-07 23:02:19 stderr File "/usr/local/lib/python3.5/dist-packages/werkzeug/serving.py", line 302, in run_wsgi
2019-07-07 23:02:19 stderr Traceback (most recent call last):
2019-07-07 23:02:19 stderr Error on request:
2019-07-07 23:02:18 stderr 172.17.0.1 - - [07/Jul/2019 23:02:18] "GET /favicon.ico HTTP/1.1" 500 -
2019-07-07 23:02:18 stdout queue full. moving on
2019-07-07 23:02:18 stdout queue full. moving on
2019-07-07 23:02:18 stderr Invalid UE golomb code
2019-07-07 23:02:18 stderr Invalid UE golomb code
2019-07-07 23:02:18 stderr 172.17.0.1 - - [07/Jul/2019 23:02:18] "GET /left/best_person.jpg HTTP/1.1" 200 -
2019-07-07 23:02:17 stdout queue full. moving on
2019-07-07 23:02:17 stdout queue full. moving on
2019-07-07 23:02:17 stdout queue full. moving on
2019-07-07 23:02:17 stdout queue full. moving on
...
2019-07-07 23:01:56 stdout queue full. moving on
2019-07-07 23:01:55 stdout queue full. moving on
2019-07-07 23:01:55 stderr [h264 @ 0x3e67040] error while decoding MB 37 117, bytestream -14
2019-07-07 23:01:55 stdout queue full. moving on
...
2019-07-07 23:01:54 stderr Invalid UE golomb code
2019-07-07 23:01:54 stderr Invalid UE golomb code
2019-07-07 23:01:54 stderr 172.17.0.1 - - [07/Jul/2019 23:01:54] "GET /back HTTP/1.1" 200 -
2019-07-07 23:01:53 stdout queue full. moving on
2019-07-07 23:01:53 stdout queue full. moving on
...
2019-07-07 23:01:48 stdout queue full. moving on
2019-07-07 23:01:48 stdout queue full. moving on
2019-07-07 23:01:48 stderr KeyError: 'favicon.ico'
2019-07-07 23:01:48 stderr frame = cameras[camera_name].get_current_frame_with_objects()
2019-07-07 23:01:48 stderr File "/opt/frigate/detect_objects.py", line 79, in imagestream
2019-07-07 23:01:48 stderr for item in iterable:
2019-07-07 23:01:48 stderr File "/usr/local/lib/python3.5/dist-packages/werkzeug/wrappers/base_response.py", line 45, in _iter_encoded
2019-07-07 23:01:48 stderr return self._next()
2019-07-07 23:01:48 stderr File "/usr/local/lib/python3.5/dist-packages/werkzeug/wsgi.py", line 507, in __next__
2019-07-07 23:01:48 stderr for data in application_iter:
2019-07-07 23:01:48 stderr File "/usr/local/lib/python3.5/dist-packages/werkzeug/serving.py", line 292, in execute
2019-07-07 23:01:48 stderr execute(self.server.app)
2019-07-07 23:01:48 stderr File "/usr/local/lib/python3.5/dist-packages/werkzeug/serving.py", line 302, in run_wsgi
2019-07-07 23:01:48 stderr Traceback (most recent call last):
2019-07-07 23:01:48 stderr Error on request:
2019-07-07 23:01:48 stderr 172.17.0.1 - - [07/Jul/2019 23:01:48] "GET /favicon.ico HTTP/1.1" 500 -
2019-07-07 23:01:48 stderr BrokenFilesystemWarning,
2019-07-07 23:01:48 stderr /usr/local/lib/python3.5/dist-packages/werkzeug/filesystem.py:60: BrokenFilesystemWarning: Detected a misconfigured UNIX filesystem: Will use UTF-8 as filesystem encoding instead of 'ascii'
2019-07-07 23:01:47 stdout queue full. moving on
2019-07-07 23:01:47 stdout queue full. moving on
2019-07-07 23:01:47 stdout queue full. moving on
...
2019-07-07 22:59:43 stdout queue full. moving on
2019-07-07 22:59:43 stdout queue full. moving on
2019-07-07 22:59:43 stdout queue full. moving on
2019-07-07 22:59:43 stderr Invalid UE golomb code
2019-07-07 22:59:43 stderr Invalid UE golomb code
2019-07-07 22:59:42 stdout queue full. moving on
2019-07-07 22:59:42 stdout queue full. moving on
2019-07-07 22:59:42 stderr Invalid UE golomb code
2019-07-07 22:59:42 stderr Invalid UE golomb code
2019-07-07 22:59:41 stdout queue full. moving on
2019-07-07 22:59:41 stdout queue full. moving on
2019-07-07 22:59:41 stdout queue full. moving on
2019-07-07 22:59:41 stdout queue full. moving on
2019-07-07 22:59:41 stderr [h264 @ 0x3e5f8e0] error while decoding MB 0 18, bytestream 67619
2019-07-07 22:59:41 stderr [h264 @ 0x3e5f8e0] left block unavailable for requested intra mode at 0 18
2019-07-07 22:59:40 stdout queue full. moving on
2019-07-07 22:59:40 stdout queue full. moving on
2019-07-07 22:59:40 stderr 172.17.0.1 - - [07/Jul/2019 22:59:40] "GET /left HTTP/1.1" 200 -
2019-07-07 22:59:39 stdout queue full. moving on
2019-07-07 22:59:39 stdout queue full. moving on
2019-07-07 22:59:39 stdout queue full. moving on
2019-07-07 22:59:39 stdout queue full. moving on
2019-07-07 22:59:39 stderr Invalid UE golomb code
2019-07-07 22:59:39 stderr Invalid UE golomb code
2019-07-07 22:59:38 stdout queue full. moving on
2019-07-07 22:59:38 stderr Invalid UE golomb code
2019-07-07 22:59:38 stderr Invalid UE golomb code
2019-07-07 22:59:35 stderr
2019-07-07 22:59:35 stderr TypeError: object of type 'NoneType' has no len()
2019-07-07 22:59:35 stderr y_location = min(int(obj['ymax']), len(self.mask)-1)
2019-07-07 22:59:35 stderr File "/opt/frigate/frigate/video.py", line 260, in add_objects
2019-07-07 22:59:35 stderr self.cameras[frame['camera_name']].add_objects(parsed_objects)
2019-07-07 22:59:35 stderr File "/opt/frigate/frigate/object_detection.py", line 55, in run
2019-07-07 22:59:35 stderr self.run()
2019-07-07 22:59:35 stderr File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
2019-07-07 22:59:35 stderr Traceback (most recent call last):
2019-07-07 22:59:35 stderr Exception in thread Thread-16:
2019-07-07 22:59:35 stderr Invalid UE golomb code
2019-07-07 22:59:35 stderr Invalid UE golomb code
2019-07-07 22:59:34 stderr Invalid UE golomb code
Here’s my config
web_port: 4000
mqtt:
host: 10.10.10.20
topic_prefix: frigate
user: MyUser
password: MyPassword
cameras:
left:
rtsp:
user: username
host: 192.168.1.20
port: 554
password: $RTSP_PASSWORD_LEFT
path: /Sms=3.unicast
mask: 1080Left.bmp
regions:
- size: 1080
x_offset: 0
y_offset: 0
min_person_area: 9000
threshold: 0.2
- size: 1080
x_offset: 0
y_offset: 839
min_person_area: 9000
threshold: 0.2
back:
rtsp:
user: username
host: 192.168.1.20
port: 554
password: $RTSP_PASSWORD_BACK
path: /Sms=4.unicast
mask: 1080Back.bmp
regions:
- size: 1080
x_offset: 0
y_offset: 0
min_person_area: 9000
threshold: 0.2
- size: 1080
x_offset: 839
y_offset: 0
min_person_area: 9000
threshold: 0.2
Based on those error messages, I think it might have something to do with your mask. Are you sure the mask file is oriented correctly and exists for both?
Okay I did find one issue in my mask… One of the files had been saved by the image editor as png rather than bmp (it likes to revert back to png even when I open a bmp so I have to be very vigilant about watching this every time I touch them). I fixed that, but I can send them to you if you want to see them…
Anyhow, now when I start it, I get bounding boxes on the video for a while, but then it gives up after a while and video goes away and all that is left in the log is this:
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
Unable to grab a frame
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
Opening the RTSP Url...
Opening the RTSP Url...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
Opening the RTSP Url...
Opening the RTSP Url...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
Opening the RTSP Url...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
Opening the RTSP Url...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
It starts out like this
On connect called
Creating a new capture process...
Starting a new capture process...
Capture process for left: 32
* Serving Flask app "detect_objects" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
Opening the RTSP Url...
queue full. moving on
queue full. moving on
queue full. moving on
queue full. moving on
queue full. moving on
queue full. moving on
queue full. moving on
with lots of queue full while it’s working and the occasional last frame is more than 2 seconds old and occasionally one of these:
Creating a new capture process...
Starting a new capture process...
Capture process for left: 30
* Serving Flask app "detect_objects" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
Opening the RTSP Url...
I wonder if opencv is just struggling to decode the portrait video feed. Ffmpeg may handle it better in the next version.
Maybe… I can try to go back to landscape… Gonna play with it a bit more and see how it does…
So I switched to the 704x576 substreams direct from the hikvision camera at 4fps and they seem to be working better, no queue full messages anymore, even with 4 streams… So maybe was something funny with the streams from the Synology or 1080p was too much to resize. Still can’t figure out what it doesn’t like about the main stream on the hikvision, but now I’m noticing that I can’t open it any more on VLC either so maybe this happened when I switched to 1080p on the camera, maybe it can’t serve up multiple 1080p streams or something… Will know by tomorrow if it stays running or slowly dies like before.
Anyhow, one more question, for some reason the MQTT availability keeps toggling on and off as shown below, any idea why that would be? Is it a timeout setting somewhere or something? It makes my history graphs rather useless. If it’s not something obvious I will just take out the availability topic config…
Check the logs for your mqtt broker.
Okay so MQTT problem was another stupid mistake on my side, I left frigate running on my old machine and it was sort of gimping along making those errors. That’s fixed now, but unfortunately the new instance against the substream petered out again after about a day, now it won’t serve up the real time feeds or recognize people… It’s strange that it worked fine with the CPU based version but not this one.
Is anyone else getting their RTSP streams from a hikvision or synology and have it working well?
Here’s the log, it’s repeating this:
Invalid UE golomb code
Invalid UE golomb code
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
Invalid UE golomb code
Invalid UE golomb code
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
last frame is more than 2 seconds old, restarting camera capture...
Terminating the existing capture process...
Creating a new capture process...
Starting a new capture process...
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code
Invalid UE golomb code