Dahua IPC to MQTT App

xbmcnut · February 13, 2019, 2:24am

Same error unfortunately.

{
  "log_level": "info",
  "system_packages": [
    "libcurl"
  ],
  "python_packages": [
    "pip==19.0.2",
    "curl-dev",
    "pycurl"
  ]
}


-----------------------------------------------------------
 aarch64 / HassOS 2.8 / HA 0.87.1 / SU 143 / stable
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing... 
Log level is set to INFO
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] 02-updates.sh: executing... 
INFO: You are running the latest version of this add-on
[cont-init.d] 02-updates.sh: exited 0.
[cont-init.d] 20-init-configuration.sh: executing... 
[cont-init.d] 20-init-configuration.sh: exited 0.
[cont-init.d] 21-compiled-dir.sh: executing... 
[cont-init.d] 21-compiled-dir.sh: exited 0.
[cont-init.d] 30-auto-token.sh: executing... 
INFO: Updating Hass.io API token in AppDaemon with the current one...
[cont-init.d] 30-auto-token.sh: exited 0.
[cont-init.d] 31-ha-url.sh: executing... 
[cont-init.d] 31-ha-url.sh: exited 0.
[cont-init.d] 50-compiled-symlink.sh: executing... 
[cont-init.d] 50-compiled-symlink.sh: exited 0.
[cont-init.d] 80-system-packages.sh: executing... 
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/aarch64/APKINDEX.tar.gz
v3.8.2-55-g801a047bfd [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]
v3.8.2-53-g53558ad6fc [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]
v3.9.0-392-g8bfa7fabdc [http://dl-cdn.alpinelinux.org/alpine/edge/main]
v3.9.0-400-g4f94de8920 [http://dl-cdn.alpinelinux.org/alpine/edge/community]
v3.9.0-400-g4f94de8920 [http://dl-cdn.alpinelinux.org/alpine/edge/testing]
OK: 22224 distinct packages available
OK: 73 MiB in 33 packages
[cont-init.d] 80-system-packages.sh: exited 0.
[cont-init.d] 81-python-packages.sh: executing... 
Collecting pip==19.0.2
  Downloading https://files.pythonhosted.org/packages/d7/41/34dd96bd33958e52cb4da2f1bf0818e396514fd4f4725a79199564cd0c20/pip-19.0.2-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-19.0.2
Collecting curl-dev
  Could not find a version that satisfies the requirement curl-dev (from versions: )
No matching distribution found for curl-dev
FATAL: Failed installing package curl-dev
[cont-init.d] 81-python-packages.sh: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] 50-compiled-symlink.sh: executing... 
[cont-finish.d] 50-compiled-symlink.sh: exited 0.
[cont-finish.d] 99-message.sh: executing...

ReneTode · February 13, 2019, 2:58am

sorry, got to go to bed now (04:00)
ill google some more tommorow if you didnt find it.

ay least your on the right track.

xbmcnut · February 13, 2019, 3:01am

Crikey! You’re worse than I am. Good night (or good morning) and thanks for your help.

algirdasc · February 13, 2019, 6:52am

Could not run curl-config: [Errno 2] No such file or directory: 'curl-config': 'curl-config'

This can be fixed by installing pycurl-dev package. Just ssh to your HassOS and run:
sudo apt-get install python-dev
If you’re running AppDaemon in docker:
docker exec -it <container_id_or_name> apt-get install python-dev

ReneTode · February 13, 2019, 12:01pm

then it probably also would work to add python-dev to the system packages

@xbmcnut 04:00 is around normal for me
but your welcome.

Velly · February 13, 2019, 2:58pm

config as: 
{
  "log_level": "info",
  "system_packages": [
    "libcurl",
    "python-dev"
  ],
  "python_packages": [
    "pip==19.0.2",
    "curl-dev",
    "pycurl"
  ]
}

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-banner.sh: executing... 
-----------------------------------------------------------
 Hass.io Add-on: AppDaemon v1.6.0
 Python Apps and HADashboard using AppDaemon 3.x for Home Assistant
 From: Community Hass.io Add-ons
 By: Franck Nijhof <[email protected]>
-----------------------------------------------------------
 armhf / HassOS 2.8 / HA 0.86.4 / SU 143 / stable
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing... 
Log level is set to INFO
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] 02-updates.sh: executing... 
INFO: You are running the latest version of this add-on
[cont-init.d] 02-updates.sh: exited 0.
[cont-init.d] 20-init-configuration.sh: executing... 
[cont-init.d] 20-init-configuration.sh: exited 0.
[cont-init.d] 21-compiled-dir.sh: executing... 
[cont-init.d] 21-compiled-dir.sh: exited 0.
[cont-init.d] 30-auto-token.sh: executing... 
INFO: Updating Hass.io API token in AppDaemon with the current one...
[cont-init.d] 30-auto-token.sh: exited 0.
[cont-init.d] 31-ha-url.sh: executing... 
[cont-init.d] 31-ha-url.sh: exited 0.
[cont-init.d] 50-compiled-symlink.sh: executing... 
[cont-init.d] 50-compiled-symlink.sh: exited 0.
[cont-init.d] 80-system-packages.sh: executing... 
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/armhf/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/armhf/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/armhf/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/armhf/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/armhf/APKINDEX.tar.gz
v3.8.2-55-g801a047bfd [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]
v3.8.2-53-g53558ad6fc [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]
v3.9.0-392-g8bfa7fabdc [http://dl-cdn.alpinelinux.org/alpine/edge/main]
v3.9.0-401-gdee82aa931 [http://dl-cdn.alpinelinux.org/alpine/edge/community]
v3.9.0-400-g4f94de8920 [http://dl-cdn.alpinelinux.org/alpine/edge/testing]
OK: 22016 distinct packages available
OK: 72 MiB in 34 packages
(1/3) Installing pkgconf (1.5.3-r0)
(2/3) Installing python2 (2.7.15-r1)
(3/3) Installing python2-dev (2.7.15-r1)
Executing busybox-1.28.4-r1.trigger
OK: 117 MiB in 37 packages
[cont-init.d] 80-system-packages.sh: exited 0.
[cont-init.d] 81-python-packages.sh: executing... 
Collecting pip==19.0.2
  Downloading https://files.pythonhosted.org/packages/d7/41/34dd96bd33958e52cb4da2f1bf0818e396514fd4f4725a79199564cd0c20/pip-19.0.2-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-19.0.2
Collecting curl-dev
  Could not find a version that satisfies the requirement curl-dev (from versions: )
No matching distribution found for curl-dev
FATAL: Failed installing package curl-dev
[cont-init.d] 81-python-packages.sh: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] 50-compiled-symlink.sh: executing... 
[cont-finish.d] 50-compiled-symlink.sh: exited 0.
[cont-finish.d] 99-message.sh: executing... 
-----------------------------------------------------------
                Oops! Something went wrong.
 
 We are so sorry, but something went terribly wrong when
 starting or running this add-on.
 
 Be sure to check the log above, line by line, for hints.
-----------------------------------------------------------
[cont-finish.d] 99-message.sh: exited 0.
[cont-finish.d] done.
[s6-finish] syncing disks.
[s6-finish] sending all processes the TERM signal.

ReneTode · February 13, 2019, 3:04pm

curl-dev might not be needed when python-dev is used so try

{
  "log_level": "info",
  "system_packages": [
    "libcurl",
    "python-dev"
  ],
  "python_packages": [
    "pip==19.0.2",
    "pycurl"
  ]
}

algirdasc · February 13, 2019, 3:53pm

I guess so. Unfortunately, I’m not familiar with HassOS.

Tuckie · February 13, 2019, 4:04pm

@algirdasc how is Dahua’s motion sensing treating you?

Have you been able to tune it’s sensitivity to your needs?

I never really gave the in-camera detection any thought, as I had assumed that hooking into a software CCTV event stream would be more robust.

algirdasc · February 13, 2019, 5:12pm

Well, speaking of motion detection the correct term would be “shitty”. Neither camera, nor NVR software could reliably detect motion. I’ve settled with in-camera motion detection to save CPU usage on my rassberry (well, it’s Odroid, but basically the same). My problem with motion detection is that one of the cameras are relatively far away from motion source, so objects (persons) in picture frame are small, thus I must set high sensitivity, which enables false detections. Other camera is in better location, but I’ve upgraded it recently from 4mpix to 6mpix, so it needs some fine tuning. Also rain / snow is not a friend when speaking of software motion detection.

What I’m currently developing, to end my motion detection hell, is camera motion detection + AI analysis. How it works - you ask? Good question. Well it works with any camera/image. But in my case, camera uploads motion frames to FTP, then my web app gets notification when it needs to process motion (In my case DahuaMQTT Motion Stop triggers HTTP request) and then my app starts fetching frames for preview, wether its files in folder or HTTP image. (I tried getting snapshot from camera, but it takes 1-10 seconds, and it is not reliable, so I settled with FTP). Then every few minutes runs AI analysis tool (DeepStack AI) and analyzes frame by frame for objects & faces. And then finally couple times a day runs python script which builds short video for preview (depends on captured frame count, but on average it is 1-3 minutes length, see 3rd screenshot) and publishes MQTT topic with download link and the rest is done via push messages. Yes, it’s bad solution for those, who wants instant motion detection, but for now (and for my needs) it seems like proven concept - I can see what happened through the day around the house and it takes only few minutes to watch it, also it is convenient to fast forward video on smartphone. Maybe after some tweaks and upgrades I’ll release it for public use.

P.S: other than that, my developed web app resizes frames to configured dimensions, extracts rectangles for object preview (see 2nd screenshot), and creates new images with marked objects for video preview (see 3rd screenshot).

See screenshot attached.

xbmcnut · February 13, 2019, 8:01pm

Thanks. Moving forward but still not there. With that config we get:

OK: 22225 distinct packages available
OK: 73 MiB in 33 packages
(1/3) Installing pkgconf (1.5.3-r0)
(2/3) Installing python2 (2.7.15-r1)
(3/3) Installing python2-dev (2.7.15-r1)
Executing busybox-1.28.4-r1.trigger
OK: 123 MiB in 36 packages
[cont-init.d] 80-system-packages.sh: exited 0.
[cont-init.d] 81-python-packages.sh: executing... 
Collecting pip==19.0.2
  Downloading https://files.pythonhosted.org/packages/d7/41/34dd96bd33958e52cb4da2f1bf0818e396514fd4f4725a79199564cd0c20/pip-19.0.2-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-19.0.2
Collecting pycurl
  Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "/tmp/pip-install-c7ko1etu/pycurl/setup.py", line 223, in configure_unix
        stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
        restore_signals, start_new_session)
      File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'curl-config': 'curl-config'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-c7ko1etu/pycurl/setup.py", line 913, in <module>
        ext = get_extension(sys.argv, split_extension_source=split_extension_source)
      File "/tmp/pip-install-c7ko1etu/pycurl/setup.py", line 582, in get_extension
        ext_config = ExtensionConfiguration(argv)
      File "/tmp/pip-install-c7ko1etu/pycurl/setup.py", line 99, in __init__
        self.configure()
      File "/tmp/pip-install-c7ko1etu/pycurl/setup.py", line 227, in configure_unix
        raise ConfigurationError(msg)
    __main__.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory: 'curl-config': 'curl-config'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-c7ko1etu/pycurl/
FATAL: Failed installing package pycurl
[cont-init.d] 81-python-packages.sh: exited 1.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] 50-compiled-symlink.sh: executing... 
[cont-finish.d] 50-compiled-symlink.sh: exited 0.
[cont-finish.d] 99-message.sh: executing... 
-----------------------------------------------------------
                Oops! Something went wrong.
 
 We are so sorry, but something went terribly wrong when
 starting or running this add-on.

algirdasc · February 13, 2019, 8:08pm

(1/3) Installing pkgconf (1.5.3-r0)
(2/3) Installing python2 (2.7.15-r1)
(3/3) Installing python2-dev (2.7.15-r1)

I see wrong version is being installed. Could you try changing from python-dev to python3-dev?

xbmcnut · February 13, 2019, 8:15pm

Thanks. Sadly, same error with this config:

{
  "log_level": "info",
  "system_packages": [
    "libcurl",
    "python3-dev"
  ],
  "python_packages": [
    "pip==19.0.2",
    "pycurl"
  ]
}

OK: 22225 distinct packages available
OK: 73 MiB in 33 packages
(1/2) Installing pkgconf (1.5.3-r0)
(2/2) Installing python3-dev (3.6.6-r0)
Executing busybox-1.28.4-r1.trigger
OK: 91 MiB in 35 packages
[cont-init.d] 80-system-packages.sh: exited 0.
[cont-init.d] 81-python-packages.sh: executing... 
Collecting pip==19.0.2
  Downloading https://files.pythonhosted.org/packages/d7/41/34dd96bd33958e52cb4da2f1bf0818e396514fd4f4725a79199564cd0c20/pip-19.0.2-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-19.0.2
Collecting pycurl
  Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "/tmp/pip-install-rmjo6otv/pycurl/setup.py", line 223, in configure_unix
        stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
        restore_signals, start_new_session)
      File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'curl-config': 'curl-config'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-rmjo6otv/pycurl/setup.py", line 913, in <module>
        ext = get_extension(sys.argv, split_extension_source=split_extension_source)
      File "/tmp/pip-install-rmjo6otv/pycurl/setup.py", line 582, in get_extension
        ext_config = ExtensionConfiguration(argv)
      File "/tmp/pip-install-rmjo6otv/pycurl/setup.py", line 99, in __init__
        self.configure()
      File "/tmp/pip-install-rmjo6otv/pycurl/setup.py", line 227, in configure_unix
        raise ConfigurationError(msg)
    __main__.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory: 'curl-config': 'curl-config'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-rmjo6otv/pycurl/
FATAL: Failed installing package pycurl

algirdasc · February 13, 2019, 8:24pm

Well, I remember strugling to install pycurl. My bash history shows, that last packages I installed are: curl-dev python3-dev libressl-dev. I think libressl-dev is not needed.

ReneTode · February 13, 2019, 8:33pm

i found this on the pycurl github:

The line of code that throws Could not run curl-config: [Errno 2] No such file or directory” when installing pycurl on Linux, should inform the user as to remedy - a pre-requisite, rather than leaving them to test their google skills. A page online is the remedy, and it could have sections like the following for Raspbian GNU/Linux 8 (jessie):
sudo apt-get install libssl-dev libcurl4-openssl-dev python-dev

i guess they didnt succeed in preventing the google hunt.
so the search goes on to the right packages.

libssl-dev
libcurl4-openssl-dev

could do the trick as system packages.

and i did some searching on egg-info errors, and it seems thats a part from the python package setuptools

so its probably to much installing, but could do the trick (remove what throws an error)

{
  "log_level": "info",
  "system_packages": [
    "libcurl",
    "python3-dev",
    "libssl-dev",
    "libcurl4-openssl-dev"
  ],
  "python_packages": [
    "pip==19.0.2",
    "setuptools",
    "pycurl"
  ]
}

algirdasc · February 13, 2019, 8:38pm

algirdas@devlaptop:~$ apt-file search curl-config | grep bin/curl-config
libcurl-openssl1.0-dev: /usr/bin/curl-config
libcurl4-gnutls-dev: /usr/bin/curl-config
libcurl4-nss-dev: /usr/bin/curl-config
**libcurl4-openssl-dev: /usr/bin/curl-config**

I guess the last one is missing.

xbmcnut · February 13, 2019, 8:39pm

Yep, didn’t like that.

OK: 22225 distinct packages available
OK: 73 MiB in 33 packages
(1/2) Installing pkgconf (1.5.3-r0)
(2/2) Installing python3-dev (3.6.6-r0)
Executing busybox-1.28.4-r1.trigger
OK: 91 MiB in 35 packages
  libssl-dev (missing):
ERROR: unsatisfiable constraints:
    required by: world[libssl-dev]
FATAL: Failed installing package libssl-dev

ReneTode · February 13, 2019, 8:43pm

oke, try again without libssl-dev

xbmcnut · February 13, 2019, 9:25pm

Had to remove that and libcurl4-openssl-dev. Sadly, same errors. I’m sure there will be some combo that works?

{
  "log_level": "info",
  "system_packages": [
    "libcurl",
    "python3-dev"
  ],
  "python_packages": [
    "pip==19.0.2",
    "setuptools",
    "pycurl"
  ]
}

Successfully installed pip-19.0.2
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages (39.0.1)
Collecting pycurl
  Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "/tmp/pip-install-jpa84dl3/pycurl/setup.py", line 223, in configure_unix
        stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
        restore_signals, start_new_session)
      File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'curl-config': 'curl-config'

algirdasc · February 13, 2019, 9:31pm

Do you have access to shell via SSH or directly?