ok, funny enough I hesitated between USB & PCIe version… I will have to either try LXC or run docker on the proxmox host instead of QEMU, will make things simpler I guess.
I see many occurrences of this on the docker host (also on the NUC itself running Debian 10… maybe a SW host or bios issue?), will investigate first:
[78140.801041] usb 3-1: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[78140.821546] usb 3-1: LPM exit latency is zeroed, disabling LPM.
I also had many issues with USB coral when I tried to pass it through to a virtualbox instance. It worked for a little while and then always ended up stopping to respond. I switched my whole setup to Unraid and now my coral runs in docker in a native Linux environment. Its now rock solid. I think there are a lot of issues with running coral in a virtual environment that for some reason creates instability.
I have this issue on the proxmox host itself. So need to fix the issue there first.
It’s a Debian 10. Coral is on usb 2-3.
root@pve:~# dmesg|grep usb
[ 0.392261] usbcore: registered new interface driver usbfs
[ 0.392261] usbcore: registered new interface driver hub
[ 0.392261] usbcore: registered new device driver usb
[ 1.449484] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.04
[ 1.449485] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.449486] usb usb1: Product: xHCI Host Controller
[ 1.449487] usb usb1: Manufacturer: Linux 5.4.65-1-pve xhci-hcd
[ 1.449487] usb usb1: SerialNumber: 0000:00:14.0
[ 1.454386] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.04
[ 1.454387] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.454389] usb usb2: Product: xHCI Host Controller
[ 1.454390] usb usb2: Manufacturer: Linux 5.4.65-1-pve xhci-hcd
[ 1.454391] usb usb2: SerialNumber: 0000:00:14.0
[ 1.455230] usb: port power management may be unreliable
[ 1.792014] usb 2-3: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 1.812370] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 1.812552] usb 2-3: New USB device found, idVendor=18d1, idProduct=9302, bcdDevice= 1.00
[ 1.812554] usb 2-3: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[ 1.944064] usb 1-4: new full-speed USB device number 2 using xhci_hcd
[ 2.098620] usb 1-4: New USB device found, idVendor=0403, idProduct=6001, bcdDevice= 6.00
[ 2.098621] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2.098622] usb 1-4: Product: EnOcean USB 300 DB
[ 2.098623] usb 1-4: Manufacturer: EnOcean GmbH
[ 2.098623] usb 1-4: SerialNumber: FTYQYFG5
[ 2.227837] usb 1-8: new full-speed USB device number 3 using xhci_hcd
[ 2.377750] usb 1-8: New USB device found, idVendor=8087, idProduct=0a2b, bcdDevice= 0.10
[ 2.377751] usb 1-8: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[ 3.705690] usbcore: registered new interface driver usbserial_generic
[ 3.705704] usbserial: USB Serial support registered for generic
[ 3.711196] usbcore: registered new interface driver ftdi_sio
[ 3.711204] usbserial: USB Serial support registered for FTDI USB Serial Device
[ 3.711255] usb 1-4: Detected FT232RL
[ 3.714078] usb 1-4: FTDI USB Serial Device converter now attached to ttyUSB0
[ 3.720564] usbcore: registered new interface driver btusb
[ 109.475882] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 109.499924] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 165.598186] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 165.618221] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 355.279099] usb 1-4: USB disconnect, device number 2
[ 862.853862] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 862.874033] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 962.937048] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 962.961011] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 1043.027628] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 1043.048074] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 1133.122165] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 1133.142172] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 1613.595778] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 1613.619991] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 1874.052685] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 1874.073055] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 1973.943670] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 1973.963636] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 2594.600690] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 2594.620772] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 3545.638256] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 3545.658557] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 3705.576473] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 3705.600674] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 3845.722946] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 3845.746934] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 4065.940496] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 4065.960339] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 4656.525566] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 4656.545593] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 4746.627366] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 4746.647251] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 4926.811406] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 4926.831766] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 5437.329688] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 5437.349726] usb 2-3: LPM exit latency is zeroed, disabling LPM.
Hey, I’ve been playing around with the TensorFlow integration but as far as I know it doesn’t take advantage of the Coral USB Accelerator so I’m trying out Frigate. I’m having some issues with false positives, but before I start troubleshooting that I’m trying to get clips to save so I can more easily troubleshoot. Unfortunately I haven’t gotten any clips as of yet; I’m hoping someone on here could give me some troubleshooting suggestions.
I’m running frigate in Docker 19.03.12 on an IBM x3650 m3. This is a Proxmox server but I have docker installed bare-metal, alongside the hypervisor. Docker compose is:
frigate:
container_name: frigate
restart: unless-stopped
privileged: true
shm_size: '1g' # should work for 5-7 cameras
image: blakeblackshear/frigate:stable
volumes:
- /dev/bus/usb:/dev/bus/usb
- /etc/localtime:/etc/localtime:ro
- /usr/share/hassio/homeassistant/frigate:/config
- /usr/share/hassio/homeassistant/www/frigate:/clips
ports:
- "5050:5000"
environment:
FRIGATE_MQTT_USER: "REDACTED"
FRIGATE_MQTT_PASSWORD: "REDACTED"
This is config.yml:
web_port: 5000
tensorflow_device: usb
mqtt:
host: REDACTED
topic_prefix: frigate
user: REDACTED
password: REDACTED
save_clips:
max_seconds: 300
objects:
track:
- person
- dog
filters:
person:
min_area: 5000
max_area: 100000
min_score: 0.5
threshold: 0.85
cameras:
backgrill:
ffmpeg:
input: REDACTED
take_frame: 1
best_image_timeout: 60
zones:
front_steps:
coordinates:
- 545,1077
- 747,939
- 788,805
filters:
person:
min_area: 5000
max_area: 100000
threshold: 0.8
save_clips:
enabled: True
pre_capture: 30
snapshots:
show_timestamp: True
draw_zones: False
objects:
track:
- person
- dog
filters:
person:
min_area: 5000
max_area: 100000
min_score: 0.5
threshold: 0.85
masterbedroom:
ffmpeg:
input: REDACTED
take_frame: 1
best_image_timeout: 60
zones:
front_steps:
coordinates:
- 545,1077
- 747,939
- 788,805
filters:
person:
min_area: 5000
max_area: 100000
threshold: 0.8
save_clips:
enabled: True
pre_capture: 30
snapshots:
show_timestamp: True
draw_zones: False
objects:
track:
- person
- dog
filters:
person:
min_area: 5000
max_area: 100000
min_score: 0.5
threshold: 0.85
A couple other things to note. I couldn’t get {FRIGATE_MQTT_USER} and {FRIGATE_MQTT_PASSWORD} working in config.yml, and the /<camera_name>/latest.jpg[?h=300] HTTP endpoint gives a 404 error.
EDIT: Also, how big of an impact would going from Coral inference delay of 10ms to 30ms be? I have a USB 3.0 PCIe card in my server and when the Coral is plugged into that I get 10ms, but when the power spikes the card shuts down. The card takes 4-pin molex power but the server has no way of supplying that. On the built-in USB 2.0 ports I get inference delays of 30ms but I don’t have the power spike issues. Would it be worth replacing the USB Accelerator with the Mini PCIe or Dual TPU M.2 card and an appropriate PCIe adapter card?
Can you post logs and the output of /debug/stats?
2020-10-02T00:44:17.167448513Z On connect called
2020-10-02T00:44:17.713861765Z ffprobe -v panic -show_error -show_streams -of json "REDACTED"
2020-10-02T00:44:17.714765480Z Starting detection process: 59
2020-10-02T00:44:19.022457622Z {'streams': [{'index': 0, 'codec_name': 'aac', 'codec_long_name': 'AAC (Advanced Audio Coding)', 'profile': 'LC', 'codec_type': 'audio', 'codec_time_base': '1/48000', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'sample_fmt': 'fltp', 'sample_rate': '48000', 'channels': 1, 'channel_layout': 'mono', 'bits_per_sample': 0, 'r_frame_rate': '0/0', 'avg_frame_rate': '0/0', 'time_base': '1/48000', 'start_pts': 0, 'start_time': '0.000000', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}, {'index': 1, 'codec_name': 'h264', 'codec_long_name': 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10', 'profile': 'Main', 'codec_type': 'video', 'codec_time_base': '1/50', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'width': 1920, 'height': 1080, 'coded_width': 1920, 'coded_height': 1080, 'has_b_frames': 0, 'sample_aspect_ratio': '1:1', 'display_aspect_ratio': '16:9', 'pix_fmt': 'yuvj420p', 'level': 41, 'color_range': 'pc', 'color_space': 'bt709', 'color_transfer': 'bt709', 'color_primaries': 'bt709', 'chroma_location': 'left', 'field_order': 'progressive', 'refs': 1, 'is_avc': 'false', 'nal_length_size': '0', 'r_frame_rate': '25/1', 'avg_frame_rate': '25/1', 'time_base': '1/90000', 'start_pts': 28800, 'start_time': '0.320000', 'bits_per_raw_sample': '8', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}]}
2020-10-02T00:44:19.023294887Z Creating ffmpeg process...
2020-10-02T00:44:19.023315923Z ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i REDACTED -f rawvideo -pix_fmt rgb24 pipe:
2020-10-02T00:44:19.029884474Z ffprobe -v panic -show_error -show_streams -of json "REDACTED"
2020-10-02T00:44:20.393214222Z {'streams': [{'index': 0, 'codec_name': 'aac', 'codec_long_name': 'AAC (Advanced Audio Coding)', 'profile': 'LC', 'codec_type': 'audio', 'codec_time_base': '1/48000', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'sample_fmt': 'fltp', 'sample_rate': '48000', 'channels': 1, 'channel_layout': 'mono', 'bits_per_sample': 0, 'r_frame_rate': '0/0', 'avg_frame_rate': '0/0', 'time_base': '1/48000', 'start_pts': 0, 'start_time': '0.000000', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}, {'index': 1, 'codec_name': 'h264', 'codec_long_name': 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10', 'profile': 'Main', 'codec_type': 'video', 'codec_time_base': '1/50', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'width': 1920, 'height': 1080, 'coded_width': 1920, 'coded_height': 1080, 'has_b_frames': 0, 'sample_aspect_ratio': '1:1', 'display_aspect_ratio': '16:9', 'pix_fmt': 'yuvj420p', 'level': 41, 'color_range': 'pc', 'color_space': 'bt709', 'color_transfer': 'bt709', 'color_primaries': 'bt709', 'chroma_location': 'left', 'field_order': 'progressive', 'refs': 1, 'is_avc': 'false', 'nal_length_size': '0', 'r_frame_rate': '25/1', 'avg_frame_rate': '25/1', 'time_base': '1/90000', 'start_pts': 32400, 'start_time': '0.360000', 'bits_per_raw_sample': '8', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}]}
2020-10-02T00:44:20.393649058Z Creating ffmpeg process...
2020-10-02T00:44:20.393667376Z ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -vsync drop -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i REDACTED -f rawvideo -pix_fmt rgb24 pipe:
2020-10-02T00:44:20.401481590Z Camera_process started for backgrill: 88
2020-10-02T00:44:20.402888983Z Starting process for backgrill: 88
2020-10-02T00:44:20.403656081Z Camera_process started for masterbedroom: 89
2020-10-02T00:44:20.405438214Z Starting process for masterbedroom: 89
2020-10-02T00:44:20.420085789Z * Serving Flask app "detect_objects" (lazy loading)
2020-10-02T00:44:20.420127512Z * Environment: production
2020-10-02T00:44:20.420134010Z WARNING: This is a development server. Do not use it in a production deployment.
2020-10-02T00:44:20.420139940Z Use a production WSGI server instead.
2020-10-02T00:44:20.420150079Z * Debug mode: off
2020-10-02T01:02:31.320828225Z Detection appears to be stuck. Restarting detection process
2020-10-02T01:02:31.320937366Z Waiting for detection process to exit gracefully...
2020-10-02T01:02:31.342082889Z Starting detection process: 404
{"backgrill":{"camera_fps":25.0,"detection_fps":5.3,"ffmpeg_pid":67,"frame_info":{"detect":1601606214.372725,"process":1601606214.372725,"read":1601606214.372725},"pid":88,"process_fps":25.0,"read_start":1601606214.378684,"skipped_fps":0.0},"coral":{"detection_start":0.0,"fps":5.3,"inference_speed":30.03,"pid":404},"masterbedroom":{"camera_fps":25.0,"detection_fps":0.0,"ffmpeg_pid":86,"frame_info":{"detect":1601606214.362849,"process":1601606214.362849,"read":1601606214.399484},"pid":89,"process_fps":25.0,"read_start":1601606214.369432,"skipped_fps":0.0},"plasma_store_rc":null}
And you get a 404 when you go to /backgrill/latest.jpg?
The page says “Not Found The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.” and the HTTP status is 404, yes. But if I go to /backgrill I see the stream,/backgrill/person/best.jpg I see the last person detected, etc. All of the endpoints but latest work.
I assume the clips problem is a permissions issue (isn’t that always the case on linux) but I didn’t see anything being logged so I wasn’t sure.
Try using the 0.6.0 tag instead of stable. Maybe you have an older version of the stable tag. Frigate should include “Camera named <camera_name> not found” in the 404 response body. If not, then make sure you dont have some kind of reverse proxy intercepting the request.
Yes, it was an old version. I originally pulled the image in July, then I tried pulling latest about a month ago and was having network issues but couldn’t troubleshoot at the time because the server was in the middle of a 2-week long HDD scan. I completely forgot to pull the latest image before I started messing with it again.
The camera/latest.jpg endpoint is indeed working properly now. I will monitor the clips folder to see if that begins working also. Thanks!
PCIe WIFI adapters typically have the right M.2 socket for this use…
Guys,
Looks like a company is making a M2 chip that will be leaps and bounds over the Coral and other devices. I don’t think its out yet but something to keep an eye on. It uses more power but it will be capable of running better NN’s and/or more cameras. If its real…
Its called “Hailo-8™ M.2 AI Acceleration Module”
https://hailo.ai/product-hailo/hailo-8-m2-module/
If it becomes available to purchase and the price isn’t crazy, I will buy you one Blake so that you could code for it and test it.
looks like a very good module and Israelis never build crap. This can solve the Deep Learning side probably if works with YoloV5 but cant solve the stream encoding/decoding. I still think xavier nx is still the best to address both “problems” Deep Learning + stream processing
What problems are you referring to? Are you just trying to save power by going smaller? Frigate in its current state is the best detector ive ever used. I have used YoloV3 and V4. They have the same results as whatever Frigate is using. I have been running Frigate on 7 cameras for a week now and have only had 2 false alerts and have not missed any cars or people.
If you are after smaller power usage then I can understand your quest but 400$ is expensive for hardware just to detect people and cars.
false detection is only one of the issues, you will miss objects with it too that you dont even notice unless test and study it. Not bad detector, but i use more cams and it needs powerful desktop PC which costs more than a xavier nx and consumes lot more electricity too. Do the math. If you dont want to miss a thing, you still need to use line crossing alert function in your cam or buy or build a separate NVR. All this routes just creating tons of costs, instead of think first and pick the right hardware. When it comes to security i do not want to give a chance to the bad guys. I use frigate and a separate Hikvision NVR, so i dont miss a thing. As i do the expensive way, i dont recommend this to you all. Blake wants Object Det with NVR function too, so his goal pretty much the same.
Hikvision NVR 300-600 USD
PC i7, 8-16GB RAM, SSD 300-400 USD
Do HW acceleration in GPU, like 1080ti second hand 400 USD
Google Coral 70 USD
Total :300+300+400+70=1070 USD minimum you can spend to have a nearly perfect system that consumes around 500-600watts of electricity. or use a better hardware solution like xavier nx.
I know many may not be in the same position, I have had great success integrating frigate into an existing vmware esxi environment.
The processors are 8 core 3.2ghz AMD with 35watt TDP which keeps power consumption low. I bought 2 usb 3.1 cards and 2 coral devices which I passthrough to 2 different virtual machines running frigate. This setup easily handles 7 hard wired cameras at 4k and the 2nd instance handles 7 wireless/power line ethernet cameras at 2k. CPU usage is around 10ghz total or 35% of the overall host cpu because of lack of hw decoding.
Now keep in mind this same host is responsible for nvme based vsan, firewall/ips/ids, provides a nfs share for cameras, cifs shares for file storage, portainer for docker management, hassio, aguard, and a few vmware horizon virtual desktop environments as well as a slew of other virtual machines.
Before I setup masks I was maxing out both of my coral devices around 90fps, but now that I have applied masks my usage is around 20-40fps. My only real question is how much hw accel via intel quicksync would have helped with as many cameras as I have. I don’t want to go to GPU accel as I think that would add too much power consumption. To be honest though the host cpu is only at 30% so there is plenty of power left. With 2 hosts, 4nvme drives, 8ssd drives and 1 hikvision nvr that has 8 cameras attached via POE the setup is only consuming 200-250watts.
For me 1 host cost only a few hundred more than the xavier nx but can do infinitely more.
Out of curiosity, is this an enterprise server? I’m asking because I have a USB3 card in my IBM x3650 m3 but the Coral seems to overdraw power and the card gets shut down. My card takes 4-pin molex power but there’s obviously no easy way to get power for that card in an enterprise server.
No, homebuilt server. I also have the cards with the 4pin molex, each getting a dedicated line from the psu. Does the motherboard have any 3pin fan connectors? Im wondering if a 3pin to 4 pin converter would work. Obviously paying close attention to power draw.
look at my messages, check USB on your host with this command:
dmesg|grep usb
Fans are modular and plug directly into motherboard with a proprietary connector. I’m thinking about tapping into the DVD drive power because I’ll never use the drive, but I’ll have to make my own cable and not sure about the current limits on that header. I found some cards on Amazon that claim to provide 2A without any direct PSU power, but I have no experience with those so I don’t know how accurate those claims are. I also don’t know if it’s worth it to go from 30ms inference on USB2 to 10ms inference on USB3.