Dahua IPC to MQTT App

I had to enter my appdaemon docker container and install packages manually.
apk add curl-dev python3-dev libressl-dev
After that I just created requirements.txt with pycurl in it. But I’m not 100% sure if thats the correct way on HassOS.

if i have understand apk correct then you dont install with apk add, but you make it available.
the install is done by the requirements file.
but i doubt that they have even apk available in hassio/hassOS/addons.

the problem is that not everything is available for such a slimmed down OS.

just an idea (dont know if its possible and if its worth it)
but you use pycurl for the connection, dont you?
isnt the same thing possible with something like requests?

Shall I try adding those?

curl-dev was already tried as python package, but maybe its a system package
i would try on or the other and remove what doesnt work.

Making some progress. Came up with a ‘gcc’ error so I added that and we move forward but not quite there yet.

{
  "log_level": "info",
  "system_packages": [
    "libcurl",
    "python3-dev",
    "curl-dev",
    "gcc"
  ],
  "python_packages": [
    "pip==19.0.2",
    "pycurl"
  ]
}

OK: 22225 distinct packages available
OK: 73 MiB in 33 packages
(1/2) Installing pkgconf (1.5.3-r0)
(2/2) Installing python3-dev (3.6.6-r0)
Executing busybox-1.28.4-r1.trigger
OK: 91 MiB in 35 packages
(1/6) Upgrading libcurl (7.61.1-r0 -> 7.61.1-r1)
(2/6) Installing libressl-dev (2.7.4-r0)
(3/6) Installing zlib-dev (1.2.11-r1)
(4/6) Installing libssh2-dev (1.8.0-r3)
(5/6) Installing nghttp2-dev (1.32.0-r0)
(6/6) Installing curl-dev (7.61.1-r1)
Executing busybox-1.28.4-r1.trigger
OK: 116 MiB in 40 packages
(1/10) Installing binutils (2.30-r5)
(2/10) Installing gmp (6.1.2-r1)
(3/10) Installing isl (0.18-r0)
(4/10) Installing libgomp (6.4.0-r9)
(5/10) Installing libatomic (6.4.0-r9)
(6/10) Installing libgcc (6.4.0-r9)
(7/10) Installing mpfr3 (3.1.5-r1)
(8/10) Installing mpc1 (1.0.3-r1)
(9/10) Installing libstdc++ (6.4.0-r9)
(10/10) Installing gcc (6.4.0-r9)
Executing busybox-1.28.4-r1.trigger
OK: 184 MiB in 50 packages
[cont-init.d] 80-system-packages.sh: exited 0.
[cont-init.d] 81-python-packages.sh: executing... 
Collecting pip==19.0.2
  Downloading https://files.pythonhosted.org/packages/d7/41/34dd96bd33958e52cb4da2f1bf0818e396514fd4f4725a79199564cd0c20/pip-19.0.2-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-19.0.2
Collecting pycurl
  Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
Installing collected packages: pycurl
  Running setup.py install for pycurl: started
    Running setup.py install for pycurl: finished with status 'error'
    Complete output from command /usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-g17t7os4/pycurl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-8yszvic9/install-record.txt --single-version-externally-managed --compile:
    Using curl-config (libcurl 7.61.1)
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-aarch64-3.6
    creating build/lib.linux-aarch64-3.6/curl
    copying python/curl/__init__.py -> build/lib.linux-aarch64-3.6/curl
    running build_ext
    building 'pycurl' extension
    creating build/temp.linux-aarch64-3.6
    creating build/temp.linux-aarch64-3.6/src
    gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Os -g -Os -g -Os -g -DTHREAD_STACK_SIZE=0x100000 -fPIC -DPYCURL_VERSION="7.43.0.2" -DHAVE_CURL_SSL=1 -DHAVE_CURL_OPENSSL=1 -DHAVE_CURL_SSL=1 -I/usr/include/python3.6m -c src/docstrings.c -o build/temp.linux-aarch64-3.6/src/docstrings.o
    In file included from src/pycurl.h:4:0,
                     from src/docstrings.c:4:
    /usr/include/python3.6m/Python.h:11:20: fatal error: limits.h: No such file or directory
     #include <limits.h>
                        ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-g17t7os4/pycurl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-8yszvic9/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-g17t7os4/pycurl/
FATAL: Failed installing package pycurl
[cont-init.d] 81-python-packages.sh: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] 50-compiled-symlink.sh: executing... 
[cont-finish.d] 50-compiled-symlink.sh: exited 0.
[cont-finish.d] 99-message.sh: executing... 
-----------------------------------------------------------
                Oops! Something went wrong.

SUCCESS Peeps!!!

Installing collected packages: pycurl
  Running setup.py install for pycurl: started
    Running setup.py install for pycurl: finished with status 'done'
Successfully installed pycurl-7.43.0.2
[cont-init.d] 81-python-packages.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
2019-02-14 13:02:42.575306 INFO AppDaemon Version 3.0.2 starting
2019-02-14 13:02:42.576080 INFO Configuration read from: /config/appdaemon/appdaemon.yaml
2019-02-14 13:02:42.579838 INFO AppDaemon: Starting Apps
2019-02-14 13:02:42.589278 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
2019-02-14 13:02:42.873772 INFO AppDaemon: HASS: HASS Plugin Initializing
2019-02-14 13:02:42.875057 INFO AppDaemon: HASS: HASS Plugin initialization complete
2019-02-14 13:02:42.876337 INFO Starting Dashboards
2019-02-14 13:02:42.898533 INFO API is disabled
2019-02-14 13:02:42.915773 INFO AppDaemon: HASS: Connected to Home Assistant 0.87.1
2019-02-14 13:02:43.283329 INFO AppDaemon: Got initial state from namespace default
2019-02-14 13:02:45.141222 INFO AppDaemon: Reading config
2019-02-14 13:02:45.148381 INFO AppDaemon: /config/appdaemon/apps/apps.yaml added or modified
2019-02-14 13:02:45.149005 INFO AppDaemon: /config/appdaemon/apps/apps.yaml added or modified
2019-02-14 13:02:45.149590 INFO AppDaemon: App 'hello_world' added
2019-02-14 13:02:45.150527 INFO AppDaemon: Adding /config/appdaemon/apps to module import path
2019-02-14 13:02:45.152064 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/hello.py
2019-02-14 13:02:45.183009 INFO AppDaemon: Initializing app hello_world using class HelloWorld from module hello
2019-02-14 13:02:45.426174 INFO hello_world: Hello from AppDaemon
2019-02-14 13:02:45.429993 INFO hello_world: You are now ready to run Apps!
2019-02-14 13:02:45.432048 INFO AppDaemon: App initialization complete

{
  "log_level": "info",
  "system_packages": [
    "libcurl",
    "python3-dev",
    "curl-dev",
    "gcc",
    "g++"
  ],
  "python_packages": [
    "pip==19.0.2",
    "pycurl"
  ]
}

Needed latest g++ compiler added. Can remove ‘gcc’ as that gets added anyway :yum:

1 Like

Getting real close!!

Here is my app.yaml file.

---
hello_world:
  module: hello
  class: HelloWorld

dahua_mqtt:
  module: dahua_mqtt
  class: DahuaMQTT

And the error;

2019-02-14 13:14:33.236980 INFO hello_world: You are now ready to run Apps!
2019-02-14 13:14:33.237597 INFO AppDaemon: Initializing app dahua_mqtt using class DahuaMQTT from module dahua_mqtt
2019-02-14 13:14:33.238487 WARNING AppDaemon: ------------------------------------------------------------
2019-02-14 13:14:33.239081 WARNING AppDaemon: Unexpected error running initialize() for dahua_mqtt
2019-02-14 13:14:33.239631 WARNING AppDaemon: ------------------------------------------------------------
2019-02-14 13:14:33.241850 WARNING AppDaemon: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/appdaemon/appdaemon.py", line 1581, in init_object
    init()
  File "/config/appdaemon/apps/dahua_mqtt.py", line 66, in initialize
    for camera in self.args["cameras"]:
KeyError: 'cameras'
2019-02-14 13:14:33.242414 WARNING AppDaemon: ------------------------------------------------------------
2019-02-14 13:14:33.244726 INFO AppDaemon: App initialization complete

Seems there is something in the code that might require customisation??

class DahuaMQTT(hass.Hass):

	proc = None
	cameras = []
	curl_multiobj = pycurl.CurlMulti()
	num_curlobj = 0
	kill_thread = False

	def initialize(self):

		for camera in self.args["cameras"]:
			dahuacam = DahuaCamera(self, camera)
			self.cameras.append(dahuacam)
			url = URL_TEMPLATE.format(**camera)

			curlobj = pycurl.Curl()
			dahuacam.curlobj = curlobj

			curlobj.setopt(pycurl.URL, url)
			curlobj.setopt(pycurl.CONNECTTIMEOUT, 30)
			curlobj.setopt(pycurl.TCP_KEEPALIVE, 1)
			curlobj.setopt(pycurl.TCP_KEEPIDLE, 30)
			curlobj.setopt(pycurl.TCP_KEEPINTVL, 15)
			curlobj.setopt(pycurl.HTTPAUTH, pycurl.HTTPAUTH_DIGEST)
			curlobj.setopt(pycurl.USERPWD, "{0}:{1}".format(camera["user"], camera["pass"]))
			curlobj.setopt(pycurl.WRITEFUNCTION, dahuacam.on_receive)

you forgot to configure in app.yaml

Example configuration:
DahuaMQTT:
  class: DahuaMQTT
  module: dahua_mqtt
  cameras:
    - host: 192.168.0.1
      port: 80
      user: user
      pass: pass
      topic: cameras/1
      events: VideoMotion,VideoBlind,VideoLoss,AlarmLocal,....
    - host: 192.168.0.2
      port: 80
      user: user
      pass: pass
      topic: cameras/2
      events: VideoMotion,VideoBlind,VideoLoss,AlarmLocal,....

glad you got pycurl finally installed

I’m a dumb arse!! I totally see the example config in the .py file now I’m looking! Thanks for that.

1 Like

Seems to be working now but despite wire cross detection configured, no MQTT events. Hmmm…

2019-02-14 13:47:09.485024 INFO AppDaemon: Reading config
2019-02-14 13:47:09.520069 INFO AppDaemon: /config/appdaemon/apps/apps.yaml added or modified
2019-02-14 13:47:09.520747 INFO AppDaemon: App 'DahuaMQTT' changed
2019-02-14 13:47:09.522550 INFO AppDaemon: Terminating DahuaMQTT
2019-02-14 13:47:09.523150 INFO AppDaemon: Calling terminate() for DahuaMQTT
2019-02-14 13:47:09.546637 INFO AppDaemon: Initializing app DahuaMQTT using class DahuaMQTT from module dahua_mqtt
2019-02-14 13:47:09.554993 INFO DahuaMQTT: [10.0.1.11] OnDisconnect(Success [])
2019-02-14 13:47:09.602785 INFO DahuaMQTT: [10.0.1.14] OnConnect()
2019-02-14 13:47:09.609108 INFO DahuaMQTT: [10.0.1.16] OnConnect()
2019-02-14 13:47:13.428498 INFO DahuaMQTT: [10.0.1.11] OnDisconnect(Success [])
2019-02-14 13:47:14.739967 INFO DahuaMQTT: [10.0.1.11] OnDisconnect(Success [])
2019-02-14 13:47:18.558975 INFO DahuaMQTT: [10.0.1.11] OnDisconnect(Success [])

Tryed too: following log
in MQTT FX topic “cameras/dahua/VideoMotion” (I changed “1” to “dahua” in apps.yaml) gives nothing.
Mqtt discovery is enabled.
Maybe I should change topic for Hassio as “homeassistant/cameras/dahua” ??
#MQTT
mqtt:
broker: 192.168.2.162
discovery: true
discovery_prefix: homeassistant
port: 1883
keepalive: 60
username: user
password: mypass

  • 1 question: is that normal that on every HASSIO reboot or AppDaemon restart action , AppDaemon do installation of all mentioned packages again? It takes so much time… Is that normal behaviour?
    And some strange happens in logs - on every few restarts it installs pip 10, and pip 19, then uninstall pip 10 etc etc etc.

      2019-02-14 17:22:48.604907 INFO AppDaemon Version 3.0.2 starting
      2019-02-14 17:22:48.605931 INFO Configuration read from: /config/appdaemon/appdaemon.yaml
      2019-02-14 17:22:48.612680 INFO AppDaemon: Starting Apps
      2019-02-14 17:22:48.626707 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
      2019-02-14 17:22:49.185514 INFO AppDaemon: HASS: HASS Plugin Initializing
      2019-02-14 17:22:49.187362 INFO AppDaemon: HASS: HASS Plugin initialization complete
      2019-02-14 17:22:49.188969 INFO Starting Dashboards
      2019-02-14 17:22:49.234088 INFO API is disabled
      2019-02-14 17:22:49.288354 INFO AppDaemon: HASS: Connected to Home Assistant 0.86.4
      2019-02-14 17:22:51.018971 INFO AppDaemon: Got initial state from namespace default
      2019-02-14 17:22:52.697659 INFO AppDaemon: Reading config
      2019-02-14 17:22:52.733463 INFO AppDaemon: /config/appdaemon/apps/apps.yaml added or modified
      2019-02-14 17:22:52.734208 INFO AppDaemon: /config/appdaemon/apps/apps.yaml added or modified
      2019-02-14 17:22:52.734912 INFO AppDaemon: App 'hello_world' added
      2019-02-14 17:22:52.735641 INFO AppDaemon: App 'DahuaMQTT' added
      2019-02-14 17:22:52.736712 INFO AppDaemon: Adding /config/appdaemon/apps to module import path
      2019-02-14 17:22:52.739136 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/hello.py
      2019-02-14 17:22:52.807824 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/dahua_mqtt.py
      2019-02-14 17:22:52.848050 INFO AppDaemon: Initializing app hello_world using class HelloWorld from module hello
      2019-02-14 17:22:53.307600 INFO hello_world: Hello from AppDaemon
      2019-02-14 17:22:53.314276 INFO hello_world: You are now ready to run Apps!
      2019-02-14 17:22:53.315391 INFO AppDaemon: Initializing app DahuaMQTT using class DahuaMQTT from module dahua_mqtt
      2019-02-14 17:22:53.322199 INFO AppDaemon: App initialization complete
      2019-02-14 17:22:54.972803 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:22:59.997791 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:05.009103 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:10.040252 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:15.049828 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:20.085555 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:25.093993 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:30.123956 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:35.133519 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:40.166384 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:45.174309 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:50.206354 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:23:55.214910 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:00.261864 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:05.287968 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:10.297348 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:15.330974 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:20.378761 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:25.413533 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:30.424347 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:35.451889 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:40.461140 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:45.490766 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:50.500577 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:24:55.509265 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:25:00.534669 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])
      2019-02-14 17:25:05.547380 INFO DahuaMQTT: [192.168.2.172] OnDisconnect(Success [])

For a moment I connected esp8266 espeasy to receive alarm out triggers from my cameras.
The only pro - is that not working (for actual moment) appdaemon solution slows down much my Hassio pi: from Config check to restart time.
To much packages maybe…

Appdaemon normally has a very low footprint.
do you have anything else in appdaemon then this app?
it also has normally NO impact on the starting time from home assistant.
so maybe you have something else going on.

Appdaemon Hello only) my problem is: full re-install of every appdaemon (python and system) package after Hassio restart

hmm, thats not as it should be, but can be hassio problem.

after that all required parts are installed you could try if you could delete the stuff in the requirements.txt
it is installed, so it shouldnt be neccesary to have it there anymore.

Sounds good, but no info on “cameras/1/VideoMotion” topic…
Log is like I quoted before
Need to ask for news @xbmcnut

if you dont get anything on mqtt, you probably dont get anything from the camera.

i dont have experience with such camera, but the app starts a loop then continiously seems to check some urls. (which by the way makes a heavy load on the RPI and your network)

i see this url
“http://{host}:{port}/cgi-bin/eventManager.cgi?action=attach&codes=%5B{events}%5D”

so you could see what data you get on that url.

Just another idea; my Hassio is SLL protected “outside” by duckdns etc , does it make any sense to trying to connect to cameras in local net(I mean cam 80 port is blocked by my router from outside requests). Maybe I should make routing ports for cam 80 port or so, and in this case it is too complicated and unsecure (some Dahua models have problems with security)
I am real dumb in docker, python and so on ,( however having few hundreds of automatisations, devices and sensors)) maybe I want to realize impossible scenario…
I can not understand clearly how script is receiving data from camera, if I could try it in local net by Get or something request, I could be more “seeing”. Now feel myself as blind… due to lack of knowledges. Good luck to others))
I am now far away from cams - could check http request by tomorrow. Thanks for! [Bought 3 bottles Scicilian wine, bought not enough…]

your camera should not be reachable from the outside at all.
hassio is local and you cam is local and appdaemon is local, so they need to connect local, and you access only 1 thing from the outside (hassio or better a proxy server)

the script uses curl to get information from this url
“http://{host}:{port}/cgi-bin/eventManager.cgi?action=attach&codes=%5B{events}%5D”
where host, port and events are replaced.

so replace host, port and events and use the url in a browser, to see if you get any result.

and indeed you need to be on your homenetwork to get such stuff wotrking correctly.