Arlo: replacement pyarlo module

Hi.

I’ve been playing around with Home Assistant for a month or so and having a lot of success but I’ve never been able to get the Arlo module to work satisfactorily - images wouldn’t appear, video streams wouldn’t work, my base stations spent a lot of time in the Unknown state, things taking too long messages…

Looking at the traces from Home Assistant any number of threads could be using pyarlo but looking at the pyarlo module it doesn’t look thread safe and it seems this can cause several potential issues:

  • pyarlo can’t handle more than one base station simultaneously. Looking at the Arlo website traces the base stations should share an event stream but the code opens one stream per base station - opening a second one closes the first.
  • In publish_and_get_event the thread opening the event stream might close it before another thread, arriving at the function later, has finished using it. It should be reference counted.
  • In publish_and_get_event the loop reading packets should wait up to 10 seconds for a response to arrive but if 2 non response packets arrive - any number of unrelated packets can arrive on an event stream - the loop will exit almost immediately. This can happen with multiple or single threads.
  • It can’t handle library videos from a deleted camera.

I started off trying to fix these issues but realised a better solution would be to make the whole module event driven. That allows it to react to changes in the background and means the functions Home Assistant uses never have to wait for network activity. I spent some time this weekend trying to get a drop in replacement and the new version:

  • does as little as possible on the start up thread
  • subscribes to all base stations at start up and opens a single event stream in a background thread
  • monitors event from Arlo and will update thumbnail images, video libraries, signal strength battery levels… in the back ground
  • has preliminary support for doorbells
  • will pickle some state across reboots

The experimental module is here: https://drive.google.com/file/d/1IjvJoSD2dCvep9k9zRqsabZdG6zOvGw1/view?usp=sharing

I use a docker to host Home Assistant and the following instruction will install the experimental package for me:

# copy py-0.3.tgz into your home assistant config directory
# assuming your instance is called home-assistant
$ docker exec -it home-assistant bash
root@skippy:/usr/src/app# cd /config/
root@skippy:/config# pip uninstall pyarlo
root@skippy:/config# pip uninstall sseclient-py
root@skippy:/config# pip install py-0.3.tgz
root@skippy:/config# exit

Then stop and restart the container:

$ docker stop home-assistant
$ docker start home-assistant

The new module is pretty chatty and you should see regular updates in the log files. It will also dump packets into /config/packets.dump so you can see what Arlo is sending you.

If it all breaks you can revert to your the original version of pyarlo with:

$ docker exec -it home-assistant bash
root@skippy:/config# pip uninstall pyarlo
root@skippy:/config# pip install pyarlo
root@skippy:/config# exit
$ docker stop home-assistant
$ docker start home-assistant

It’s very alpha, it’s working for me but please back up your data before trying or just bring up a new Home Assistant instance. My immediate plan is to tidy the code up - I can see the tracker.py file disappearing, fill in the missing interfaces and to add motion and audio detection support with callbacks. The problem I have right now is it’s so cold here (-25C) that all my cameras have shut down.

This is only my second time posting here so I apologise if I broke any site rules. Also, it wasn’t my intent to step on any toes, I just had some spare time last week and wanted to figure out the Arlo issues I was seeing.

Thanks,

Steve

12 Likes

Thanks for your effort to improve the Arlo module. I will follow your work closely but will not be able to test the code until it reaches at least a stable beta.

Thanks. And I realise re-reading the instructions it was a bit of a big ask.

I’ve rolled it all into a single custom component - called aarlo so it won’t interfere with current arlo installs - to make installation easier. I’ll test it for a few more days then seek out some volunteers.

I wouldn’t mind another Arlo solution, I am experiencing the same issues as you’ve noted above with my setup as well.

Let me know where to pick up the custom component. Would like to actually see the battery level for once.

The custom component is available here:

To install:

cd where-you-put-your-config/custom_components
tar xf <where-you-downloaded-the-file>/arlo-0.1.tgz

Change your config to use aarlo instead of arlo, this was deliberate so I can tell it’s actually using the new components. For example, on my system:

configuration.yaml

aarlo:
  username: !secret arlo_username
  password: !secret arlo_password
  scan_interval: 300

alarm_control_panel:
  - platform: aarlo
    home_mode_name: Home
    away_mode_name: Armed

camera:
  - platform: aarlo
    ffmpeg_arguments: '-pred 1 -q:v 2'

sensors.yaml

- platform: aarlo
  monitored_conditions:
  - last_capture
  - total_cameras
  - battery_level
  - captured_today
  - signal_strength

and also try binary_sensors.yaml

- platform: aarlo                                                                                                                   
  monitored_conditions:                                                                                                             
  - motion                                                                                                                          
  - sound  

Then stop and start you home assistant installation.

A few things:

  • it’s an alpha release but has been working well for me for the last few days
  • it’s chatty but that will help me debug
  • you can try adding ding to the binary sensor monitored conditions but I haven’t had time to test my doorbells fully
  • it won’t work with non standard cameras, I will fix that tonight
  • in lovelace you will see streaming/recording and idle states in the camera tiles

Thanks for looking,

Steve

1 Like

I moved everything to github. Instructions to install are in the README.

3 Likes

Thanks for your efforts. Working well! Not really a programmer, but would like to contribute if I can (even a coffee!)

Thanks a lot! This is greatly appreciated! Works just fine here with 2 base stations and 5 cameras. And I’m running Firefox.

I only had some problems to get the custom lovelace card running at first. Some of the attributes (battery, signal_strength) didn’t work for me. The log file indicated “battery is undefined” etc.

Thanks - this looks indeed promising - just installed it … the only issue I have with this it doesnt seem to support the Arlo Baby cam fully yet - had a browse through the files and can see you look for some details of these and it seems to work partially but temperature’, ‘humidity’, ‘air_quality’ doesnt seem to work and it doesnt show a control panel as the regular one did either .

I did notice that I can disarm from the control panel but arming doesnt seem to work in my setup.

I also got an error still though installed as advised:

2019-02-11 12:04:10 ERROR (MainThread) [homeassistant.setup] Error during setup of component arlo
Traceback (most recent call last):
File “/srv/homeassistant/lib/python3.6/site-packages/homeassistant/setup.py”, line 148, in _async_setup_component
component.setup, hass, processed_config) # type: ignore
File “/usr/lib/python3.6/concurrent/futures/thread.py”, line 56, in run
result = self.fn(*self.args, **self.kwargs)
File “/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/arlo.py”, line 48, in setup
conf = config[DOMAIN]
KeyError: ‘arlo’
2019-02-11 12:04:10 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of alarm_control_panel.arlo. Setup failed for dependencies: arlo
2019-02-11 12:04:10 ERROR (MainThread) [homeassistant.setup] Unable to prepare setup for platform alarm_control_panel.arlo: Could not set up all dependencies.

I also get plenty of these:

2019-02-11 15:52:10 INFO (ArloBackgroundWorker) [pyaarlo] fast refresh
2019-02-11 15:52:10 WARNING (ArloBackgroundWorker) [pyaarlo] file not written

Hi and thanks so much for this - for the best most is working fine apart from the Lovelace component. Adding the custom is filling the logs the error. Not sure what I have missed?

https://unpkg.com/@polymer/[email protected]/lib/utils/async.js?module:44:11 Uncaught TypeError: Cannot read property ‘state’ of undefined

@grantc

I can reproduce the Uncaught Type error by asking for sound detection on a camera that doesn’t support it - one of the original Arlo cameras. Try:

  • removing sound from the show field of the Lovelace card or
  • make sure you have sound in the binary_sensor configuration

I do need to improve the attribute checking in the card, I’ll look into this this week.

@r4nd0m

Thanks for trying and for the feedback.

I’ll take a look at the baby cam stuff. I don’t have one here but if it worked in the original component I should be able to figure out why it’s not working in mine.

As for the backtrace:

  • Did you install from the github, the backtrace is showing a trace for arlo inside site-packages but the code now installs in custom_components or
  • did you leave a component as arlo rather than rename it to aarlo?

For the arming: can you look (grep) for new-mode in the logs, it should appear when you try to change mode.

For the excessive logging: I’ll quieten those down… they really should be debug or removed all together…

@ffm777

Can you paste in the full log line? It could be one of several things.

Thanks.

I installed like this:

source /srv/homeassistant/bin/activate
pip uninstall pyarlo
pip uninstall sseclient-py
pip install py-0.3.tgz

downloaded from your provided link same as as the aarlo package - then just used the ./install go /my_directory/

with regards to still loading “arlo” - had a dead one in my configuration.yaml still - usually I use packages so this should have loaded it …

no “new mode” messages in my logs at all … but just removed the arlo - will restart and report back … maybe they conflict at this stage …

edit: updated - arlo error is gone as expected … but no change with regards to being able to arm from HASS

the entity shows like this:

code_format: null
changed_by: null
attribution: Data provided by arlo.netgear.com
device_id: [ removed ]
friendly_name: BASE
icon: mdi:security

Thanks for your attention to this component, @sherrell! Have you had a chance to check out this Arlo library? It was developed separately from PyArlo, but it is a bit more robust and actively maintained. https://github.com/jeffreydwalter/arlo

I’m just wondering if there is opportunity for integration vs. reinventing the wheel for the Xth time. It may already have a lot of the features and device compatibility you are looking for. Would appreciate your thoughts!

Here you go:

2019-02-11 08:50:32 ERROR (MainThread) [frontend.js.latest.201901211] http://192.168.178.53:8123/local/aarlo-glance.js:147:8 TypeError: battery is undefined

I installed manually by copying the files into /config/custom_components and the custom card into /config/www.

@ffm777

Can you make sure you have battery_level against the monitored_conditions in the aarlo sensor configuration. You should have some sensors named sensor.aarlo_battery_level_camera_name in the States page. If you can’t see a sensor like that then can you tell me what kind of camera this is?

But, looking at the card code I can and will make it more robust to hide the error.

@jwillaz

(These are just my thoughts, I didn’t mean to step on anyone’s toes by doing this!)
I looked at both PyArlo and Arlo. My original plan was to fix some bugs in PyArlo to address problems I was seeing - for example, unknown base station states - and I was moving code from Arlo into PyArlo to speed up the process. But I realised even with the bug fixes there were still issues:

  • it was reactive not proactive - each time home-assistant asked for an update it would query the arlo servers
  • it opened and closed the event stream far too often, almost once per query, and the code to reuse the stream wasn’t thread safe
  • it didn’t support motion or audio events (or doorbells)

The Arlo library addressed a lot of these issues but had it’s own issue:

  • if you listen for events on 2 base stations on the same account they will keep logging each other out of the event stream
  • I would have to write my own event handler

In the end I smooshed the 2 libraries together - learning a lot about python along the way - taking the best pieces from both, and tried to make it all work a little more like the official Arlo web site - open the event stream once, keep it open and dispatch events to interested parties as they arrive.

Hope that goes a little way to explaining.

@r4nd0m

Many apologies but you don’t need to do the pip uninstall/pip install anymore. If you clone the git repository it contains all the python modules and you just need to run install.

What state is your alarm_control_panel showing?

actually I dont think it does much of a difference - as I downloaded the master branch and performed the install go … no changes unfortunately …

as previously mentioned:

alarm_control_panel.aarlo_base disarmed code_format: null changed_by: null attribution: Data provided by arlo.netgear.com device_id: [ removed ] friendly_name: BASE icon: mdi:security

I’ll be away for a week now - so no rush :slight_smile: