Amcrest IP Camera Component Enhancements - PTZ control and audio streaming

Third party integrations
181

Remote site: Hass.io 93.1 RasPi3B - 3 physical cameras, logger defaulting to warn.

  • old Dahua IPC bullet - POE
  • new Amcrest IP2M-851 - wifi
  • new Amcrest IP2M-842 - wifi

I have two separate camera tabs - one high resolution (HD) for local access, and the other low resolution (SD)for remote access. This is because the uplink speed at the remote site is pretty bad. I recall a conversation where this might cause issues later on, but unless there’s a new way to dynamically select resolution, I hope to stick with it until it breaks.

I rarely see comm errors on the Dahua POE camera, and as you can see from the logs, they are only appearing for the wifi-attached Amcrest cameras. The wifi signal strength was fine the last time I measured it and may not have anything to do with the comm errors.

I was using small Lovelace “badges” to represent each camera, thinking that would be quicker. But actually it was not. The badges slowly painted line by line from top to bottom, or not at all. It was the badges that didn’t paint at all that showed up in the logs as comm errors.

By switching to “picture entity” cards, the first time the page was accessed, invariably one or more cards failed to display (comm errors), but refreshing the screen, they would appear and be cached for quick re-display the next time this page was accessed.

I haven’t played around yet with the new error options, but will do so now. I do see though that I need to add back to my logging to disable all these urrlib3 warnings. (I thought that was going to be done in this new code).

In case you’re interested, here’s the log for the past hour or so…

2019-05-18 15:06:05 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for amcrest which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-05-18 15:06:05 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for custom_updater which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-05-18 15:06:09 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for life360 which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-05-18 15:06:22 WARNING (MainThread) [homeassistant.components.camera] Setup of platform amcrest is taking over 10 seconds.
2019-05-18 15:20:22 WARNING (SyncWorker_7) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=1
2019-05-18 15:20:22 WARNING (SyncWorker_3) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:20:22 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image LakeCam SD camera due to error: CommError
2019-05-18 15:20:22 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image DriveCam HD camera due to error: CommError
2019-05-18 15:21:15 WARNING (SyncWorker_10) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:21:15 WARNING (SyncWorker_2) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:21:15 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image DriveCam HD camera due to error: CommError
2019-05-18 15:21:15 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image LakeCam HD camera due to error: CommError
2019-05-18 15:25:29 WARNING (SyncWorker_2) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:25:29 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image LakeCam HD camera due to error: CommError
2019-05-18 15:25:30 WARNING (SyncWorker_9) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:25:30 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image DriveCam HD camera due to error: CommError
2019-05-18 15:26:16 WARNING (SyncWorker_14) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=1
2019-05-18 15:26:16 WARNING (SyncWorker_8) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:26:16 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image DriveCam SD camera due to error: CommError
2019-05-18 15:26:16 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image LakeCam HD camera due to error: CommError
2019-05-18 15:31:17 WARNING (SyncWorker_17) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:31:17 WARNING (SyncWorker_11) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:31:17 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image LakeCam HD camera due to error: CommError
2019-05-18 15:31:17 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image DriveCam HD camera due to error: CommError
2019-05-18 15:32:55 WARNING (SyncWorker_4) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:32:55 WARNING (SyncWorker_11) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:32:55 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image DriveCam HD camera due to error: CommError
2019-05-18 15:32:55 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image LakeCam HD camera due to error: CommError
2019-05-18 15:46:24 WARNING (SyncWorker_5) [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0
2019-05-18 15:46:24 ERROR (MainThread) [custom_components.amcrest.camera] Could not get image LakeCam HD camera due to error: CommError
182

Wait, are you saying you have multiple config entries for the same camera (e.g., “LakeCam SD” & “LakeCam HD”)? That’s not going to work. No wonder you’re seeing so many snapshot errors.

These cameras don’t seem to be able to respond to multiple, simultaneous snapshot commands (like they seem to be able to for most other commands.) The snapshot command serialization is only done for each HA camera entity individually, not amongst a set of HA camera entities for the same physical camera. If you do have multiple HA camera entities for the same physical camera, then if they try to grab a snapshot at the same time (i.e., one tries while another one is already being processed), then that will definitely cause these errors. There was no intention to handle this type of scenario.

So far I’m only filtering urllib3 errors that happen at startup (when the authorization is being determined), or once error recovery mode starts (and it’s sending a command it doesn’t normally use to “ping” the camera periodically.) It’s really only intending to reduce log clutter at startup for cameras that aren’t connected (or bad configs), and when something happens that causes a working camera to not work anymore (it gets turned off, the network goes out, etc.) The problem with trying to filter other urllib3 warnings is, at least as far as I’ve figured out so far, you have to use the warning info provided by the log system to decide whether or not to let the message through or to filter it out. Since there’s potentially a lot of other network traffic from other components I have to be careful not to filter out stuff from other components. Also, I think it’s good to see warnings, which in theory, shouldn’t be happening all the time. But, I’m just getting started and maybe there’s more that I can do.

183

Regarding filtering urllib3 - I’m perfectly fine adding urrlib3.connectionpool: error back to logger to get rid of all those warnings, including for other components.

I no longer have a page with both the HD and SD camera’s together. They are separate sections now. So wouldn’t you expect that when accessing the “SD-Remote” camera page, that only 1 snapshot request per camera would be issued? Just having another page with the HD cameras defined that wasn’t accessed won’t hurt will it?

BTW - the original log showed that had 70 comm errors, there were camera badge icons for both HD and SD variants together. That’s gone now.

Have you ever looked into the feasibility being able to select either the main (HD) or secondary (SD) stream? EDIT: dynamically

EDIT#2: I also had snapshot quality set to 5, and will change that to 3. Heading out now - will check back with you later after more testing. Thanks.

184

FWIW, that works with home-assistant.log (and console output, which is usually seen via journalctl, e.g.,), but not for the Info page. After a couple of discussions on discord, I discovered the only way to filter those out is via adding a logging.Filter. So, yeah, you can do that, but it won’t stop urllib3 warnings from filling up the Info page.

I think you can clearly see the answer in the log you showed. Notice it’s getting snapshots (or getting errors while attempting to get snapshots) for both SD & HD. Besides, that’s not the only issue. Each camera entity thinks it’s in total control of a physical camera. If one does something (like turn it on or off, etc.) the other one won’t know that happened. Again, this component was not written to share a physical camera in that way. Yes, you can probably grab streams from multiple clients, and some other things might work if you’re careful and know what you’re doing, but you’re basically asking for trouble having a HA amcrest camera entity interacting with a physical camera while anything else might be doing that at the same time (such as another HA amcrest camera entity pointing at the same physical device.)

Look, you can do whatever you want if it works. “I don’t care what anything was designed to do. I care about what it can do.” :slight_smile:

No, but that sounds like a good idea. I can add it to the list. From the amcrest component perspective, I suspect that would be pretty easy to do. Just not sure how the rest of the system might react in all scenarios.

185

Okay, I’m running on a pi3b+ using HA 0.93.1on HassOS 2.11 with an amcrest IP camera model IMP-721S. Reading back on this thread I am getting conflicted between what files are needed where and the 404 errors when trying to go to them.
Please describe what files I need, where do I get them and what do I need to configure to download this and assist you.

186

Do you mean to help test the new error handling? If so, you need to be comfortable with getting files from github and copying them onto your system. Basically, follow this link to my github fork of homeassistant. You’ll see a bunch of files there. You’ll need to copy those files to a folder you’ll need to create in the HA config directory named custom_components/amcrest. Then you should just be able to restart HA and it should use those instead of the version of the component that comes with HA.

187

Done… I’ll keep in touch.

1 Like
188

Okay, I’m exercising it now. The multiple and annoying constant warning errors have disappeared. I do get an occasional

Could not get image 406_east camera due to error: ReadTimeoutError

but I am blaming my cheap router which is being replaced with a new/better/more expensive one this week.

189

That’s great. Thanks!

There were quite a few changes, including in the low level pypi.org package, but I’ve been working on this for a while, so I’m starting to forget some of the details. lol! One of the changes that I think helped was changing the max retry count from 3 to 1 as well as changing the timeout from 3 to 3.05. (The 3.05 is actually recommended in the documentation for the requests library because it’s slightly larger than some TCP/IP timeout.) I also did away with the duplicate retrying in the pypi.org code. (I.e., only the networking packages will do retrying.) All together I think it actually helps avoid warnings and errors, but certainly cuts down on the number of messages when something does go wrong (such as a network glitch.)

Anyway, I don’t claim it’s perfect, and I’m willing to fine tune it some more if needed. But it’s good to hear it helps (but mostly that I haven’t made things worse! :slight_smile:)

190

and the snapshot function is now flawless… dunno if that was something you worked on or just coincidence. BTW, the last notification of a snapshot was just of barn swallows coming home to nest and not the cat I am still waiting to see arrive at the front door.

1 Like
191

I guess it depends on how old of a version you were using. I did make a change quite some time ago (like two months ago) to serialize shapshot commands, but that came out in HA 0.90.0.

192

@pnbruckner - thanks for the response. As part of my troubleshooting the errors, I’ve tried a variety of options (Direct component as Amcrest in HA, Generic in HA, Zoneminder, and Surveillance Station). Currently have HA set up with the 4 cameras and 2 of them via Synology Surveillance Station (for testing that as alternative solution).

Cameras - Four Amcrest IP3M-HX2. Three wireless and one wired. All four are configured the same way except for the camera name and IP.

  • Main Stream - H.264H, 2304x1296, 20FPS, CBR, 4096 Bit Rate, 40 Frame Internal
  • Sub Stream - H.264H, 640x480, 30FPS, CBR, 512 Bit Rate, 60 Frame Internal
  • Snapshot - General, 2304x1296, Quality 5, Interval 1S

HA Version - 0.93.1 in Docker on a Synology NAS.

Camera config within HA - Currently have the 4 amcrest added under an amcrest.yaml file (sample shown in my original post) and then also have the following set up in the main configuration.yaml.

ffmpeg:

camera:
  - platform: amcrest

  - platform: synology
    url: https://10.0.0.100:5001
    username: !secret synology_user
    password: !secret synology_password
    timeout: 15
    verify_ssl: False

stream:

Current Issues and Errors

"Could not get image from XYZ CAMERA due to error: Could not communicate with camera." due to "components/amcrest/camera.py (ERROR)

"Query failed due to error 401 Client Error: Unauthorized for url: http://10.0.0.XYZ:80/cgi-bin/snapshot.cgi?channel=0" due to "/usr/local/lib/python3.7/site-packages/amcrest/http.py (ERROR)"

"Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /cgi-bin/snapshot.cgi?channel=0" due to "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py (WARNING)"

"Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='10.0.0.179', port=80): Read timed out. (read timeout=3.0)")': /cgi-bin/snapshot.cgi?channel=0" due to "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py (WARNING)"

Additional Notes

  • Removed “camera_view: live” from the Picture Entity Cards in the UI since it was freezing HA and making it unresponsive (had to restart to remove those lines from UI).
  • This puts me into a state where 2 of the cameras show no image (just broken image icon, camera name, and “streaming”) and 2 of them that show a snapshot every 10 minutes. The 2 that show the snapshot are also connected to Synology Surveillance Station, but the snapshot is coming from the direct feed, NOT Synology. Also, the 2 cameras NOT connected to Synology do not show the circle bubble preview when listed within a group card in the HA UI.

Primary Goal - get snapshots restored and remove unnecessary errors.
Secondary Goal - figure out live view in HAUI. May require me to move HA to a new device due to hardware issues that are causing the freezing/unresponsive UI issues. And then get streaming to other devices like Google Home Hub, etc.

Any recommendations? Should I try updating the Amcrest files to see if that helps? Appreciate it.

193

So I guess it’s still a bit unclear to me if any of these four cameras are being controlled or accessed by anything else besides the HA amcrest component. If they are, then the first thing I would do, at least as a test, is to only use them with the HA amcrest component (i.e., stop having anything else access them in any way) and see if that has any effect. I wouldn’t access them with anything else until you get the HA side figured out. It’s possible that multiple systems interacting with a given physical camera simultaneously is part or all of the problem.

Specifically to the point above, is the following interacting with a camera that is also configured in the amcrest component?

  - platform: synology
    url: https://10.0.0.100:5001
    username: !secret synology_user
    password: !secret synology_password
    timeout: 15
    verify_ssl: False

If so then temporarily comment that out.

Next I’d try reducing the snapshot quality to 4 or less, again, just temporarily as a test. I have an IP3M-HX2 connected via WiFi and it’s setup is actually identical to yours and it’s working fine. But still it might be worth a try with a lower quality setting as a test.

You certainly can try the enhanced error handling. There’s a reply a few back where I explained how to install them. If you do try that I’d be very interested to see if that helps your scenario.

BTW, you don’t need - platform: amcrest under camera:. That hasn’t been required for quite some time. Honestly I don’t even know what effect that has, if any. I suspect it’s effectively ignored.

194

hehehe, I just looked at my own config. I also had the - platform: amcrest lingering in my cameras. I removed it without effect. You’re assumption seems valid that it was just being ignored.

195

Yeah, now that I think about how it works some more, I know that although it has no negative side effect (other than taking up space and a few CPU cycles), it’s completely ignored.

196

BTW, this also contains a new feature, which is to control the camera’s indicator light. It will be turned on if either the audio or video stream is enabled, and it will be turned off when both are disabled. On many Amcrest cameras it doesn’t make much difference because the indicator light is not in a very visible location. But on some (e.g., the IP3M-HX2, where it’s right on the front) it is visible. This will help let you know if your camera is outputting either audio or video, and hence can be heard/viewed somewhere.

I’ve actually submitted a separate PR with this feature, which also includes the ability to disable it if you don’t want it for some reason.

Anyway, just thought I’d mention this, and as usual, I’m interested in feedback, good or bad.

197

excellent work, Phil. keep it up, the effort does not go unnoticed! :slight_smile:
HA is throwing curveballs with all these breaking changes in the custom component department. A few of my custom components broke completely… guess I should have read the warnings that have been floating around logs for the last few releases…

1 Like
198

Even though I have a little button thingie to toggle on/off the mic on my picture_entity card displaying my amcrest camera, and it does indeed display a toggle on/off response… I assume the audio broadcasts via HA is not there yet. (assuming I am using a Chrome browser)
Or did I miss something in my configuration?

199

I have yet to get sound via HA streaming. I know the audio control works, because if I access the camera via Amcrest’s software, the sound does go on and off in response to the amcrest_enable/disable_audio services. I honestly haven’t looked into why the sound doesn’t come out via HA streaming. I just assumed that was something to do with the stream component. But I suppose it might have to do with sound settings in the camera. Maybe the default codec isn’t supported by HA’s streaming???

200

I think I’ve finally gotten to the bottom of the “401 Client Error: Unauthorized…” issues!

The short version

Retrying was being done at the (very low) urllib3 level, but error 401 handling (required for Digest Authentication) is done at the (not quite so low) requests level, so the error was basically being improperly handled (mainly during simultaneous snapshot commands), and hence, improperly logged.

The fix

I’m updating the amcrest pypi.org package to remove retrying at the urllib3 level and putting it back in at this package’s “application” level. I’ll release a new version of that, then update the HA amcrest error handling branch.

Done. See code here.

The long version

Digest Authentication always requires two transactions for every camera command, even after the authorization header is determined at startup. The first sends the command with the authorization header, to which the camera always responds with an error 401, but also with details needed to send the command again successfully. Mainly this consists of a “nonce” (a one-time use number needed for cryptography.) Then the command is sent again, but this time with additional headers calculated from the nonce, and (generally) the command works. This is just how Digest Authentication works.

And generally it does work for most commands, even if being sent simultaneously from multiple “clients.”

But (at least) the snapshot is different. If a snapshot command is currently being processed (i.e., the response has not yet been completely returned), if a new snapshot command comes in, although the first of the two transactions will happen normally, it will not respond to the second transaction.

With the fix in place this will result in (at the amcrest package level):

Query failed due to error: ConnectionError(ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)),)

Which will be simplified to just “Could not get image from XYZ camera due to error: CommError” at the amcrest HA level.

But before the fix, the urllib3 library would attempt to retry after the ConnectionError. But it would do so with headers that were calculated (by the requests library above it) based on the nonce that has already effectively been used. So the camera would reject it with an error 401, requesting the client calculate new headers based on a new nonce. But since this retrying was being done at the urllib3 level, which doesn’t know how to handle error 401, it would just raise the error 401 up to the requests library. But the requests library thought it already did the second part of the Digest Authentication protocol, so it just punted the error 401 up to its client, i.e., the amcrest package, which would in turn punt it up to the HA amcrest component.

So, bottom line, the main issue was the urllib3 library was trying to handle an error it did not have the context, or implementation, to handle, because it was being used incorrectly. The fix is basically to not have the urllib3 library try to do that, but rather pass it back to the requests library that has the context and knowledge to properly process it.

1 Like