Tapo cameras, frigate, go2rtc - I cannot figure out how to pass audio

Configuration
1

I have set up a couple of tapo c320ws cameras with frigate like this, as indicated by the frigate docs:

go2rtc:
  streams:
    tapo_test: 
      - rtsp://xx:xx@ip:554/stream1 
      - "ffmpeg:tapo_test#audio=opus"
    tapo_test_sub: 
      - rtsp://xx:xx@ip:554/stream2 

cameras:
  tapo_test:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/tapo_test 
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/tapo_test_sub 
          input_args: preset-rtsp-restream
          roles:
            - detect
    live:
        stream_name: tapo_test

I have audio when watching the video in the frigate UI, when the dropdown is set to webrtc
image

I have video feed but without audio in feed that is passed through to my main home assistant machine. Same for the recordings in Frigate, correct video but without audio.

Here is the info screen from the go2rtc page:

{
  "producers": [
    {
      "type": "RTSP source",
      "url": "rtsp://IP:554/stream1/",
      "remote_addr": "IP:554",
      "user_agent": "go2rtc/1.2.0",
      "medias": [
        "video, sendonly, 96 H264/90000",
        "audio, sendonly, 8 PCMA/8000"
      ],
      "tracks": [
        "96 H264/90000, sinks=1",
        "8 PCMA/8000, sinks=0"
      ],
      "recv": 2934365
    },
    {
      "url": "ffmpeg:tapo_test#audio=opus"
    }
  ],
  "consumers": [
    {
      "type": "RTSP client",
      "url": "rtsp://127.0.0.1:8554/tapo_test",
      "remote_addr": "127.0.0.1:48408",
      "user_agent": "FFmpeg Frigate/0.12.1-367d724",
      "medias": [
        "video, recvonly, 96 H264/90000",
        "audio, recvonly, 0 MPEG4-GENERIC/0"
      ],
      "tracks": [
        "96 H264/90000, sinks=1"
      ],
      "send": 2404263
    }
  ]
}

It seems to like the MPEG4 codec, although input is in pcma.
I have also tried "ffmpeg:tapo_test#audio=pcma" which does not work unfortunately.

Adding the codecs at the end of the stream name like this:

      inputs:
        - path: rtsp://127.0.0.1:8554/tapo_test?video=h264&audio=pcma

This works momentarily, i see the correct stream in the go2rtc info screen but frigate keeps crashing.

Has anyone managed to set it up, or can help me out looking at my code?

2

mp4 recordings need AAC audio, and so does home assistant. In this case you should configure your stream:

go2rtc:
  streams:
    tapo_test: 
      - rtsp://xx:xx@ip:554/stream1 
      - "ffmpeg:tapo_test#audio=aac"
    tapo_test_sub: 
      - rtsp://xx:xx@ip:554/stream2
3

Thank you!!!
I spend all morning with trial and error and it was so simple <3

4

I have two follow-up questions, if you could help :slight_smile:

  1. I tried to rotate the detector view, for a camera that is positioned on the side. From what i found online i tried this:
go2rtc:
  streams:
    tapo_test: 
      - rtsp://xxx:xxx@IP:554/stream1
      - "ffmpeg:tapo_test#audio=aac" 
    tapo_test_sub: 
      - rtsp://xxx:xxx@IP:554/stream2
      - "ffmpeg:tapo_test_sub#video=h264#rotate=90"

But it does not seem to rotate the view, at least not in the debug view. Any ideas?

  1. I noticed that in my frigate instance, go2rtc’s version is 1.2.0 while the go2rtc addon is on 1.5.0.
    Should i be bothered to upgrade it? or will installing both cause problems?
5

How would you expect go2rtc to know when to send the standard stream vs when to send the rotated stream? In short, you can’t, so you need to create a separate stream for the rotated view to query it specifically.

You can use a custom version if you want to Advanced Options | Frigate

6

I thought this was what i was doing here:

    tapo_test_sub: 
      - rtsp://xxx:xxx@IP:554/stream2
      - "ffmpeg:tapo_test_sub#video=h264#rotate=90"

This works (but need to configure the new rotated resolution)

    tapo_test_sub: 
      - rtsp://xxx:xxx@IP:554/stream2
    tapo_test_sub_r90: 
      - "ffmpeg:tapo_test_sub#video=h264#rotate=90"

Thanks!

Thanks!

7

No, that is one stream with two different inputs

1 Like
8

This will be better:

    tapo_test_sub_r90: 
      - "ffmpeg:rtsp://xxx:xxx@IP:554/stream2#video=h264#rotate=90"

This one will never work. Because the codec of the first and second source is the same (H264). And if the client requests this codec, go2rtc will choose the first source with this codec.

    tapo_test_sub: 
      - rtsp://xxx:xxx@IP:554/stream2
      - "ffmpeg:tapo_test_sub#video=h264#rotate=90"
2 Likes
9

Thank you very much, I had the same problem and I got it thanks to your help, but I only have audio if I change the mode to webrtc, in mse I can’t activate audioo, is it normal? thanks Captura de pantalla 2023-08-02 a las 11.58.54

10

that’s because in go2rtc 1.2 (current version included in frigate) aac audio is required for mse. In the next version of frigate which uses go2rtc 1.6.0 it won’t be the case.