As long as FFmpeg can read the UDP stream, you should be able to use it. The default input params assume tcp.
The stream looks fine in VLC player (using UDP) but hereās what the same stream looks like view with frigate
Vlc isnāt using ffmpeg under the hood. See if you can get ffplay working.
how do I use ffplay?
hereās the command running in the frigate container
ffmpeg -loglevel verbose -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -rtsp_transport udp -i rtsp://user:[email protected]:554/cam/realmonitor?channel=1&subtype=1 -f rawvideo -pix_fmt rgb24 pipe:
are there any switches you can think of that may help the video?
And an Atomic Pi will be able to be used with a Coral Stick?
As a VMware specialist, Iām just thinking if I want to buy a bigger system and run some VMs there taking over some tasks that are now running on different PIsā¦ Would be cool to have some VMware at home, too haha. But will be a lot of work and I suspect it to be more power consumingā¦ we will see.
The Atomic Pi works with the USB3 Coral version. It gets 16ms inference speeds just like an RPi4, but it has a x86 architecture which makes it easier to get hardware accelerated decoding to work in ffmpeg. In my opinion, it is the best option for a dedicated low power board if you only have a couple of cameras. If you have lots of cameras, you will be better off with a NUC or something that gets you <10ms inference times.
Personally, I decided to move away from a power hungry server setup with a bunch of VMs. I now run all of my home automation on a cluster of low TPU servers with K3s. Way less overhead than a VM.
1 Like
ffplay takes most of the same parameters as ffmpeg. Try this:
ffplay -rtsp_transport udp -i rtsp://home:[email protected]:554/cam/realmonitor?channel=1&subtype=1
I start Frigate with a coral and it appears to work for a little while, but without fail, after a certain amount of time, it starts throwing the errors below and eventually my VM ubuntu instance freezes and then I have to restart. After restart it does the same thing, works for a while then errors and freezes.
Any ideas?
2020-07-01T20:43:00.489983892Z On connect called,
2020-07-01T20:43:01.240516788Z ffprobe -v panic -show_error -show_streams -of json "rtsp://user:pass@ipaddress:554/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif",
2020-07-01T20:43:01.241202123Z Starting detection process: 22,
2020-07-01T20:43:07.934194162Z {'streams': [{'index': 0, 'codec_name': 'h264', 'codec_long_name': 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10', 'profile': 'Main', 'codec_type': 'video', 'codec_time_base': '0/2', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'width': 704, 'height': 480, 'coded_width': 704, 'coded_height': 480, 'has_b_frames': 0, 'sample_aspect_ratio': '0:1', 'display_aspect_ratio': '0:1', 'pix_fmt': 'yuv420p', 'level': 22, 'chroma_location': 'left', 'field_order': 'progressive', 'refs': 1, 'is_avc': 'false', 'nal_length_size': '0', 'r_frame_rate': '25/1', 'avg_frame_rate': '0/0', 'time_base': '1/90000', 'start_pts': 18000, 'start_time': '0.200000', 'bits_per_raw_sample': '8', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}, {'index': 1, 'codec_type': 'data', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'r_frame_rate': '0/0', 'avg_frame_rate': '0/0', 'time_base': '1/90000', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}]},
2020-07-01T20:43:07.934670355Z Creating ffmpeg process...,
2020-07-01T20:43:07.934678333Z ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://user:pass@ipaddress:554/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif -f rawvideo -pix_fmt rgb24 pipe:,
2020-07-01T20:43:07.940371849Z Camera_process started for garage_bullet: 35,
2020-07-01T20:43:07.940969851Z Starting process for garage_bullet: 35,
2020-07-01T20:43:07.951282431Z * Serving Flask app "detect_objects" (lazy loading),
2020-07-01T20:43:07.951298406Z * Environment: production,
2020-07-01T20:43:07.951382859Z WARNING: This is a development server. Do not use it in a production deployment.,
2020-07-01T20:43:07.951388832Z Use a production WSGI server instead.,
2020-07-01T20:43:07.951447141Z * Debug mode: off,
2020-07-01T21:00:00.688729338Z Traceback (most recent call last):,
2020-07-01T21:00:00.688981328Z File "detect_objects.py", line 361, in <module>,
2020-07-01T21:00:00.688990216Z main(),
2020-07-01T21:00:00.688993687Z File "detect_objects.py", line 164, in main,
2020-07-01T21:00:00.689005704Z client.connect(MQTT_HOST, MQTT_PORT, 60),
2020-07-01T21:00:00.689015661Z File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 937, in connect,
2020-07-01T21:00:00.692450350Z return self.reconnect(),
2020-07-01T21:00:00.692464197Z File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1071, in reconnect,
2020-07-01T21:00:00.692864622Z sock = self._create_socket_connection(),
2020-07-01T21:00:00.692871992Z File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3522, in _create_socket_connection,
2020-07-01T21:00:00.693853884Z return socket.create_connection(addr, source_address=source, timeout=self._keepalive),
2020-07-01T21:00:00.693862813Z File "/usr/lib/python3.7/socket.py", line 728, in create_connection,
2020-07-01T21:00:00.694470361Z raise err,
2020-07-01T21:00:00.694479361Z File "/usr/lib/python3.7/socket.py", line 716, in create_connection,
2020-07-01T21:00:00.694891611Z sock.connect(sa),
2020-07-01T21:00:00.694900976Z ConnectionRefusedError: [Errno 111] Connection refused,
Do you always get that error in the logs? That error looks like your VM loses the connection to the MQTT server and is unable to reconnect. Almost like the network drops out for the VM. I donāt have any ideas other than running in a VM seems to be problematic in several scenarios.
Yes, this is the only error in the frigate container log that I am seeing. I will try to figure out if the network is dropping out. It is odd that it only seems make a difference if the coral is attached when frigate is running.
Double checked everything, but as far as I can see nothing else is using the Coral.
Iām running Ubuntu within Proxmox, would it help to reboot the hole machine?
Mmmm Iām spotting the following messages in my dmesg:
[1002744.246909] xhci_hcd 0000:01:1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 13
[1002744.246995] xhci_hcd 0000:01:1b.0: Looking for event-dma 0000000024e70d00 trb-start 00000000682338f0 trb-end 0000000000000000 seg-start 0000000068233000 seg-end 0000000068233ff0
[1002744.246996] xhci_hcd 0000:01:1b.0: Looking for event-dma 0000000024e70d00 trb-start 0000000024e70000 trb-end 0000000024e70b50 seg-start 0000000024e70000 seg-end 0000000024e70ff0
[1002762.151526] usb 7-1: reset SuperSpeed USB device number 4 using xhci_hcd
[1002762.171925] usb 7-1: LPM exit latency is zeroed, disabling LPM.
I think usb 7-1 is my Coral:
$ lsusb
Bus 007 Device 004: ID 18d1:9302 Google Inc.
Bus 007 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
@blakeblackshear After about 2 weeks running 5.1, Frigate encounter issue. The docker logs shows following:
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 1 more times
TrackedObjectProcessor: unable to connect plasma client
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 80 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 79 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 78 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 77 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 76 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 75 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 74 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 73 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 72 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 71 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 70 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 69 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 68 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 67 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 66 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 65 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 64 more times
/arrow/cpp/src/plasma/io.cc:168: Connection to IPC socket failed for pathname /tmp/plasma, retrying 63 more times
I thought this might be a kernel issue with the older ubuntu 18.04. So I did a fresh installation of 20.04, but now I get the following error message:
frigate_1 | ffprobe -v panic -show_error -show_streams -of json "rtsp://192.168.*.*:*/*"
frigate_1 | Starting detection process: 22
frigate_1 | On connect called
frigate_1 | {'streams': [{'index': 0, 'codec_name': 'aac', 'codec_long_name': 'AAC (Advanced Audio Coding)', 'codec_type': 'audio', 'codec_time_base': '1/11025', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'sample_fmt': 'fltp', 'sample_rate': '11025', 'channels': 1, 'channel_layout': 'mono', 'bits_per_sample': 0, 'r_frame_rate': '0/0', 'avg_frame_rate': '0/0', 'time_base': '1/11025', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}, {'index': 1, 'codec_name': 'h264', 'codec_long_name': 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10', 'codec_type': 'video', 'codec_time_base': '0/2', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'width': 0, 'height': 0, 'coded_width': 0, 'coded_height': 0, 'has_b_frames': 0, 'sample_aspect_ratio': '0:1', 'display_aspect_ratio': '0:0', 'level': -99, 'chroma_location': 'left', 'refs': 1, 'is_avc': 'false', 'nal_length_size': '0', 'r_frame_rate': '90000/1', 'avg_frame_rate': '0/0', 'time_base': '1/90000', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}]}
frigate_1 | Traceback (most recent call last):
frigate_1 | File "detect_objects.py", line 361, in <module>
frigate_1 | main()
frigate_1 | File "detect_objects.py", line 204, in main
frigate_1 | frame_shape = get_frame_shape(ffmpeg_input)
frigate_1 | File "/opt/frigate/frigate/video.py", line 47, in get_frame_shape
frigate_1 | frame_shape = frame.shape
frigate_1 | AttributeError: 'NoneType' object has no attribute 'shape'
ffprobe is returning 0 for the width and height of your video stream. Try specifying the width and height parameters for that camera like in the example config.
After many recompiles of my Dockerfile (which is an ugly, disorderly mess right now)ā¦ I have compiled v0.5.1 on the Raspberry Pi 4. Only running 1 cam so far on this versionā¦ so havenāt gathered any stats yet.
Also, not running Raspbian Buster 32bit - I am running the Ubuntu 20.04 LTS Arm64 OS image.
Iām now running two VMās on my Intel NUC, if I run the command below on both machines I get different results:
docker run --rm --privileged --shm-size=1g -v /dev/bus/usb:/dev/bus/usb -v /var/lib/docker/volumes/frigate_data/_data/:/config:ro -v /etc/localtime:/etc/localtime:ro -p 5000:5000 blakeblackshear/frigate:stable ffprobe -v panic -show_error -show_streams -of json "rtsp://192.168.*.*:*/*"
The new Ubuntu 20.04 returns:
"width": 0,
"height": 0,
The old Ubuntu 18.04 returns:
"width": 640,
"height": 360,
How is that even possible with Docker? Both running the same images, even compared the sha256.
I added the width and height to the config, frigate is running again. But still I see these messages:
Detection appears to be stuck. Restarting detection process
Waiting for detection process to exit gracefully...
Starting detection process: 62
Not sure whatās wrong here 
Containers share a common kernel. Not sure what FFmpeg depends on in the kernel, but I have definitely seen hardware acceleration support disappear if the container was built with a different kernel than the host machine is running.
