Anki Vector integration

Hey Ryan,

This looks great but won’t work with escape pod, only the go sdk works with it as shown here: Escape Pod: Changelog, Limitations & Known Issues - Digital Dream Labs Knowledge Base
Any clues on how to change the sdk?

Hi GreyJackal,

If you use Cyb3rdog workaround for the SDK, it should in theory work with it but I haven’t tried that yet… I know the latest Escape Pod unfortunately breaks a few things with 3rd party extensions so believe you’ll have to stick with V1.

GitHub - cyb3rdog/escapepod_python_sdk: The EscapePod Python SDK for Cyb3rVector’s EscapePod Extension Proxy

Hey Ryan. Is this still working for you? I get an error when I try to do it:

ImportError: Error loading shared library ld-linux-armhf.so.3: No such file or directory (needed by /usr/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-arm-linux-gnueabihf.so)
[15:30:24] FATAL: Failed executing init command: python3 -m anki_vector.configure
cont-init: info: /etc/cont-init.d/appdaemon.sh exited 1
cont-init: warning: some scripts exited nonzero

@kaylasiemon
I haven’t tested it recently since HA has updated python to 3.10, looks like that is the issue due to cyb3rdogs SDK only supports up to 3.9.

1 Like

Only get 3 entities now…

Authenticating with Anki Cloud... DONE
Attempting to download guid from Vector-xxxx at 192.168.1.28:443... DONE
Writing config file to '/root/.anki_vector/sdk_config.ini'...
SUCCESS!
cont-init: info: /etc/cont-init.d/appdaemon.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun appdaemon (no readiness notification)
s6-rc: info: service legacy-services successfully started
[21:14:42] INFO: Starting AppDaemon...
2022-10-02 22:14:49.391186 INFO AppDaemon: AppDaemon Version 4.2.1 starting
2022-10-02 22:14:49.391569 INFO AppDaemon: Python version is 3.10.5
2022-10-02 22:14:49.391853 INFO AppDaemon: Configuration read from: /config/appdaemon/appdaemon.yaml
2022-10-02 22:14:49.392229 INFO AppDaemon: Added log: AppDaemon
2022-10-02 22:14:49.392621 INFO AppDaemon: Added log: Error
2022-10-02 22:14:49.393121 INFO AppDaemon: Added log: Access
2022-10-02 22:14:49.393489 INFO AppDaemon: Added log: Diag
2022-10-02 22:14:49.666237 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
2022-10-02 22:14:50.283146 INFO HASS: HASS Plugin Initializing
2022-10-02 22:14:50.283590 INFO HASS: HASS Plugin initialization complete
2022-10-02 22:14:50.284775 INFO AppDaemon: Initializing HTTP
2022-10-02 22:14:50.286380 INFO AppDaemon: Using 'ws' for event stream
2022-10-02 22:14:50.294727 INFO AppDaemon: Starting API
2022-10-02 22:14:50.300652 INFO AppDaemon: Starting Admin Interface
2022-10-02 22:14:50.301472 INFO AppDaemon: Starting Dashboards
2022-10-02 22:14:50.334188 INFO HASS: Connected to Home Assistant 2022.9.7
2022-10-02 22:14:50.361635 INFO AppDaemon: App 'vector_test' added
2022-10-02 22:14:50.363654 INFO AppDaemon: Found 1 total apps
2022-10-02 22:14:50.364678 INFO AppDaemon: Starting Apps with 1 workers and 1 pins
2022-10-02 22:14:50.366506 INFO AppDaemon: Running on port 5050
2022-10-02 22:14:50.396339 INFO AppDaemon: New client Admin Client connected
2022-10-02 22:14:50.467488 INFO HASS: Evaluating startup conditions
2022-10-02 22:14:50.486163 INFO HASS: Startup condition met: hass state=RUNNING
2022-10-02 22:14:50.486935 INFO HASS: All startup conditions met
2022-10-02 22:14:50.545005 INFO AppDaemon: Got initial state from namespace default
2022-10-02 22:14:52.386018 INFO AppDaemon: Scheduler running in realtime
2022-10-02 22:14:52.400392 INFO AppDaemon: Adding /config/appdaemon/apps to module import path
2022-10-02 22:14:52.405377 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/vector.py
2022-10-02 22:14:53.474408 WARNING AppDaemon: No app description found for: /config/appdaemon/apps/hello.py - ignoring
2022-10-02 22:14:53.476570 INFO AppDaemon: Initializing app vector_test using class VectorWorld from module vector
2022-10-02 22:14:53.495037 INFO AppDaemon: App initialization complete
2022-10-02 22:16:24.953582 INFO AppDaemon: Client disconnection from Admin Client
2022-10-02 22:19:53.223064 INFO vector_test: Vector Status...
2022-10-02 22:19:53.227258 WARNING vector_test: ------------------------------------------------------------
2022-10-02 22:19:53.228109 WARNING vector_test: Unexpected error in worker for App vector_test:
2022-10-02 22:19:53.228938 WARNING vector_test: Worker Ags: {'id': '93fd326a91e946e78d858fe88267a035', 'name': 'vector_test', 'objectid': 'b156cd419a7649ca8a26929e6a3bd1c1', 'type': 'scheduler', 'function': <bound method VectorWorld.vector_status of <vector.VectorWorld object at 0x7fb9e2ebc0>>, 'pin_app': True, 'pin_thread': 0, 'kwargs': {'interval': 300, '__thread_id': 'thread-0'}}
2022-10-02 22:19:53.229594 WARNING vector_test: ------------------------------------------------------------
2022-10-02 22:19:53.233378 WARNING vector_test: Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/appdaemon/threading.py", line 904, in worker
    funcref(self.AD.sched.sanitize_timer_kwargs(app, args["kwargs"]))
  File "/config/appdaemon/apps/vector.py", line 24, in vector_status
    with anki_vector.Robot("xxxxxxx",behavior_control_level=None) as robot: #Your Vector Serial Number
  File "/usr/lib/python3.10/site-packages/anki_vector/robot.py", line 758, in __enter__
    self.connect(self.behavior_activation_timeout)
  File "/usr/lib/python3.10/site-packages/anki_vector/robot.py", line 646, in connect
    self.conn.connect(timeout=timeout)
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 483, in connect
    raise e
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 496, in _connect
    self._control_events = _ControlEventManager(self._loop)
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 86, in __init__
    self._granted_event = asyncio.Event(loop=loop)
  File "/usr/lib/python3.10/asyncio/locks.py", line 168, in __init__
    super().__init__(loop=loop)
  File "/usr/lib/python3.10/asyncio/mixins.py", line 17, in __init__
    raise TypeError(
TypeError: As of 3.10, the *loop* parameter was removed from Event() since it is no longer necessary
2022-10-02 22:19:53.234052 WARNING vector_test: ------------------------------------------------------------
2022-10-02 22:22:26.300097 INFO AppDaemon: New client Admin Client connected
2022-10-02 22:24:53.030793 INFO vector_test: Vector Status...
2022-10-02 22:24:53.036032 WARNING vector_test: ------------------------------------------------------------
2022-10-02 22:24:53.037248 WARNING vector_test: Unexpected error in worker for App vector_test:
2022-10-02 22:24:53.038400 WARNING vector_test: Worker Ags: {'id': '93fd326a91e946e78d858fe88267a035', 'name': 'vector_test', 'objectid': 'b156cd419a7649ca8a26929e6a3bd1c1', 'type': 'scheduler', 'function': <bound method VectorWorld.vector_status of <vector.VectorWorld object at 0x7fb9e2ebc0>>, 'pin_app': True, 'pin_thread': 0, 'kwargs': {'interval': 300, '__thread_id': 'thread-0'}}
2022-10-02 22:24:53.039680 WARNING vector_test: ------------------------------------------------------------
2022-10-02 22:24:53.041361 WARNING vector_test: Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/appdaemon/threading.py", line 904, in worker
    funcref(self.AD.sched.sanitize_timer_kwargs(app, args["kwargs"]))
  File "/config/appdaemon/apps/vector.py", line 24, in vector_status
    with anki_vector.Robot("xxxxxxx",behavior_control_level=None) as robot: #Your Vector Serial Number
  File "/usr/lib/python3.10/site-packages/anki_vector/robot.py", line 758, in __enter__
    self.connect(self.behavior_activation_timeout)
  File "/usr/lib/python3.10/site-packages/anki_vector/robot.py", line 646, in connect
    self.conn.connect(timeout=timeout)
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 483, in connect
    raise e
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 496, in _connect
    self._control_events = _ControlEventManager(self._loop)
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 86, in __init__
    self._granted_event = asyncio.Event(loop=loop)
  File "/usr/lib/python3.10/asyncio/locks.py", line 168, in __init__
    super().__init__(loop=loop)
  File "/usr/lib/python3.10/asyncio/mixins.py", line 17, in __init__
    raise TypeError(
TypeError: As of 3.10, the *loop* parameter was removed from Event() since it is no longer necessary
2022-10-02 22:24:53.042755 WARNING vector_test: ------------------------------------------------------------

Same issue, only 3 entities, no cam

2023-01-19 23:00:00.024194 WARNING vector_test: ------------------------------------------------------------
2023-01-19 23:00:00.026720 WARNING vector_test: Unexpected error in worker for App vector_test:
2023-01-19 23:00:00.027728 WARNING vector_test: Worker Ags: {'id': '0a634d3b2f4e4fc8a2f82cf1df94f73a', 'name': 'vector_test', 'objectid': '1024e361a1c749d8be424b392d371e5c', 'type': 'scheduler', 'function': <bound method VectorWorld.vector_status of <vector.VectorWorld object at 0x7fa52eca90>>, 'pin_app': True, 'pin_thread': 1, 'kwargs': {'interval': 300, '__thread_id': 'thread-1'}}
2023-01-19 23:00:00.028325 WARNING vector_test: ------------------------------------------------------------
2023-01-19 23:00:00.031980 WARNING vector_test: Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/appdaemon/threading.py", line 904, in worker
    funcref(self.AD.sched.sanitize_timer_kwargs(app, args["kwargs"]))
  File "/config/appdaemon/apps/vector.py", line 24, in vector_status
    with anki_vector.Robot("00902a3f",behavior_control_level=None) as robot: #Your Vector Serial Number
  File "/usr/lib/python3.10/site-packages/anki_vector/robot.py", line 758, in __enter__
    self.connect(self.behavior_activation_timeout)
  File "/usr/lib/python3.10/site-packages/anki_vector/robot.py", line 646, in connect
    self.conn.connect(timeout=timeout)
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 483, in connect
    raise e
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 496, in _connect
    self._control_events = _ControlEventManager(self._loop)
  File "/usr/lib/python3.10/site-packages/anki_vector/connection.py", line 86, in __init__
    self._granted_event = asyncio.Event(loop=loop)
  File "/usr/lib/python3.10/asyncio/locks.py", line 168, in __init__
    super().__init__(loop=loop)
  File "/usr/lib/python3.10/asyncio/mixins.py", line 17, in __init__
    raise TypeError(
TypeError: As of 3.10, the *loop* parameter was removed from Event() since it is no longer necessary
2023-01-19 23:00:00.032575 WARNING vector_test: ------------------------------------------------------------

With the new piper / faster-whisper stuff is it plausible to add these inclusions to this process so we can have two-way communication with vector and home assistant?

Also, will the stuff above work with vector when he is not escape podded?

That’s cool. But it seems like this pulls information from the DDL cloud, rather than something like Wirepod or Escapepod.

@Hellman_x

Yes, It needed DDL servers to gain access to vector.
Now that the DDL servers are gone this won’t work anymore.

Can the code be refactored to work with wire-pod?

@NonaSuomy
I’m sure it can be done but it will require someone with the needed skills. It’s not that simple coz the python library will need updating too.

The good thing about it is that now it’s only needs to work with wirepod.l and no need to make it compatible with ddl anymore.

I would even be happy if someone could just make a wirepod GO addon that just dumps what you said to vector to the HA assist web API as that would be less susceptible to failure points like python versions etc. The Vector stats are cool. Much nicer to get him to do real work in HA. Command Assist would probably be the easiest and Notifications Services sent to wirepod for him to say stuff would be the next best.

I’ve been trying to make a Wire-Pod add-on for HA, I’ve got it running but I am not that familiar with HA add-ons / Docker.
I need to try and work out how to properly use the persistent storage within the Docker environment so that when HA is rebooted, the stored settings are not lost.
I think it might be the case that a bash script is written to copy over and copy back the relevant data files for Wire-Pod, before the chipper script is started.



For anyone that wants to give it a go, familiarise yourself with HA local add-on, how to FTP in to HA or create files within the addons folder.

Create a folder called wire-pod inside the addons directory

Create a Dockerfile inside /addons/wire-pod/, with the following:

ARG BUILD_FROM=ubuntu:jammy
# hadolint ignore=DL3006
FROM ${BUILD_FROM}

# Edit your timezone here:
ENV TZ=Europe/London
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

# Install software
RUN apt-get update \
 && apt-get install -y \
    sudo \
    nano \
    git

# Setup Sudoers
RUN adduser --disabled-password --gecos '' wirepod
RUN adduser wirepod sudo
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers

# Switch to created wirepod user
USER wirepod

# Create /wire-pod directory
RUN sudo mkdir /wire-pod
RUN sudo chown -R wirepod:wirepod /wire-pod
RUN cd /wire-pod

# Download wire-pod
RUN git clone https://github.com/kercre123/wire-pod/ wire-pod

# Build wire-pod
WORKDIR /wire-pod
RUN chmod a+x ./setup.sh
RUN chmod a+x ./chipper/start.sh
RUN sudo STT=vosk ./setup.sh

# Start chipper
WORKDIR /wire-pod/chipper
CMD sudo /wire-pod/chipper/start.sh

Create a config.yaml inside /addons/wire-pod/, with the following:

---
name: Wire-Pod
version: 1.0
slug: wire-pod
description: Docker Ubuntu base for WirePod
url: https://github.com/hassio-addons/addon-ubuntu-base
webui: http://[HOST]:[PORT:8080]
arch:
  - aarch64
  - amd64
  - armv7
homeassistant_api: true
uart: true
ports:
  443/tcp: 443
  80/tcp: 80
  8080/tcp: 8080

Create a build.yaml inside /addons/wire-pod/, with the following:

---
build_from:
  aarch64: arm64v8/ubuntu:jammy
  amd64: amd64/ubuntu:jammy
  armv7: arm32v7/ubuntu:jammy

Give it a few minutes and then install within the ‘Add-on Store’ under local add-on

3 Likes

Thanks for sharing Ryan,

Hopefully I can try this out next weekend…

Happy new year :tada:

1 Like

Have you had a go at making “go” scripts for vector?

So somehow I was able to accomplish this :exploding_head: with codepilot! Hope it helps get a leg up for someone who wants to do more :smiley:

Feel free to include it in your add-on @RGarrett93

Now all we need is an HA TTS integration for him!

Edit

I was able to access Vectors TTS like this:

2 Likes

Great work, I also tried to create such add-on since 2 weeks, but without success.
With your description I was able to bring it up&running on my RPi under HAOS. :+1:

Unfortually, Vector cannot find server because it look’s like hostname wouldn’t be published to my local network. Do you have any hint on this?
Beside of this, I’m already researching how to use persistent storage for this.

1 Like

I recently switched to using LXC as I was modifying the docker too much. Yes persistent storage would be a lot better for not losing code/settings if you update the docker container.

For docker you will need to look at how to add a docker container to a bridge so that it grabs the IP from your router and not the docker subnet. ( I was using the docker-net-dhcp plugin. On newer versions of docker you may have to look for an issue on their tracker about freezing on boot which I was in and list how to compile it to get it working properly. ) but there are other things like macvlan ipvlan etc. (I was having issues with them where they can’t communicate to the host properly that is why I was using the plugin)

Maybe you can try with LXC if you want. Modify this setup to get it on your proper network. IE look at your nic name with ip addr and the arch type if you are using arm etc, change it below etc.

/etc/netplan/xx-someautogen.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
    enp3s0:
      dhcp4: no
  bridges:
    br0:
      dhcp4: yes
      interfaces:
        - enp3s0
sudo apt install lxc
sudo lxc init
Would you like to use LXD clustering? (yes/no) [default=no]: no
Do you want to configure a new storage pool? (yes/no) [default=yes]: yes
Name of the new storage pool [default=default]: default
Name of the storage backend to use (btrfs, ceph, dir, lvm, zfs) [default=zfs]:
dir
Would you like to connect to a MAAS server? (yes/no) [default=no]: no
Would you like to create a new local network bridge? (yes/no) [default=yes]: yes
What should the new bridge be called? [default=lxdbr0]: lxdbr0
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: auto
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: auto
Would you like LXC to be available over the network? (yes/no) [default=no]: yes
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]: yes
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: yes
sudo lxc-create -t download -n wirepod -- --dist ubuntu --release jammy  --arch amd64
sudo lxc-ls
wirepod
sudo lxc-stop -n wirepod
sudo vim /var/lib/lxc/wirepod/config
# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = linux64

# Container specific configuration
lxc.rootfs.path = dir:/var/lib/lxc/wirepod/rootfs
lxc.uts.name = wirepod
lxc.start.auto = 1

# Network configuration
lxc.net.0.type = veth
lxc.net.0.link = br0
lxc.net.0.flags = up
lxc.net.0.hwaddr = de:ad:be:ef:ca:fe
sudo lxc-start -n wirepod
sudo lxc-attach -n wirepod
git clone https://github.com/kercre123/wire-pod --depth=1
cd wire-pod
sudo STT=vosk ./setup.sh
./setup.sh daemon-enable
systemctl start wire-pod
systemctl status wire-pod
1 Like

Unfortunately vector will only connect to a server named ‘escapepod’