HikSink: An ultra light Hikvision camera event to MQTT bridge

CornerBit · September 26, 2021, 11:54am

Hi all,

I’m releasing HikSink, a Hikvision camera event to MQTT bridge I wrote with a focus on Home Assistant automation integration. Hikvision cameras have had good in-built object and motion detection for a while, but there was no robust and easy way to act on this information e.g. to turn on lights and send alerts. The integrations I’ve tried have either silently failed after a few weeks or required that the camera’s authentication standards be lowered.

HikSink is an open-source light-weight program that listens to these events and publishes them to an MQTT server. Some key HikSink features are:

Easy Camera Authentication:
Supports digest authentication, the default on Hikvision cameras, so you don’t need to lower your security settings.
Full Event Type Support:
Automatically learns event types that HikSink doesn’t yet know about.
Home Assistant Integration:
Supports the Home Assistant MQTT discovery protocol. Provides offline notifications for individual cameras and the bridge.
Failure hardened:
Connections to cameras lost due to network issues or exceptions are automatically reestablished.
Low CPU and memory footprint:
Written in Rust and requires less than 20 MB of memory. Can easily run on a Raspberry Pi.

For example, these are the entities discovered on a DS-2CD2185FWD IP camera automatically loaded into Home Assistant:

I’ve personally run this for a few months without issue so I thought i’d share it publicly. Hopefully it helps some of you with your projects!

Installation:

Install instructions are in the main repo but it’s really just one step when using docker. Be sure to grab a copy of the sample config file and modify it with your camera and MQTT details. Then just run the following:

docker run -d \
    --name=hiksink \
    --restart=unless-stopped \
    -v <path/to/config.toml>:/app/config.toml \
    cornerbit/hiksink:latest

Please let me know if you’ve got any questions, suggestions, or issues! Also, if you find an interesting use case please share it here

florian84z · September 27, 2021, 5:26pm

Your app sounds great. I came across it through a search. I ordered two Hikvision Acesense cameras today. I would like to integrate your app into iobroker, that should also be possible via MQTT, right? Unfortunately I don’t get your repo installed in my Docker environment, I’m not that experienced with Docker / Linux either. The Config file is on / mnt / user / appdata / HikSink. What is the correct command of your repo?

rindlerblabla · September 27, 2021, 8:10pm

Cool! Tried that out to see if my ColorVu camera broadcasted some info about person/vehicle detection, which is supported by the camera, but obviously it doesn’t.

Don’t know if it’s a bug, but immediately after a motion is detected it flickers to unavailable and then straight to cleared. This make the history of the entities a bit hard to read since the log is spammed with those messages.

CornerBit · September 28, 2021, 7:37am

I run it with an Acusense camera which works well for me. I hadn’t heard of IOBroker, but from a skim of their docs I think it would work. You should definitely give it a go .
You need to download the sample configuration file somewhere on your server (e.g. /mnt/user/appdata/HikSink/config.toml). You can do this by running the following (change the path to whatever works for you):

wget -O /mnt/user/appdata/HikSink/config.toml https://raw.githubusercontent.com/CornerBit/HikSink/f102e1e261d6fa9dfb9848c1b2ce63c02ef9c575/sample_config.toml

Modify that config.toml file with your MQTT and camera details and then you can run the docker command:

docker run -d \
    --name=hiksink \
    --restart=unless-stopped \
    -v /mnt/user/appdata/HikSink/config.toml:/app/config.toml \
    cornerbit/hiksink:latest

Let me know if you have any issues getting that working

CornerBit · September 28, 2021, 7:44am

Hmm, I think the camera is sending a response HikSink can’t understand so it terminates the connection and reconnects. Unfortunately Hikvision is a bit inconsistent with their API across different hardware models and I haven’t yet been able to get my hands on a ColorVu camera.

Can you check the log output of HikSink? It will output error messages whenever this happens. I’d like to add support for the ColorVu cameras so if you can send me that part of the log I’d greatly appreciate it .

rindlerblabla · September 28, 2021, 7:30pm

Will copy some logs and send DM.

jshank · September 28, 2021, 11:06pm

Really great solution, I had worked on something similar for quite a while but never published it, good on you!

I have 3 camera’s configured and they are showing up in HA but all of the Entities for each camera are shown as Unavailable.

Device info
DS-2CD2335FWD-I (IPCamera)
by Hikvision
Firmware: Camera Firmware V5.5.61 (build 180718)

Entities
Driveway CH1 Attended Baggage Unavailable
Driveway CH1 Bad Video Unavailable
Driveway CH1 Disk Error Unavailable
Driveway CH1 Disk Full Unavailable
Driveway CH1 Face Detection Unavailable
...

Some of the logs, let me know if you want the rest via github:

DEBUG Camera coms{camera=Driveway id=driveway}: hyper::client::connect::http: connected to 10.1.10.27:80
DEBUG hyper::proto::h1::io: flushed 377 bytes
 INFO Camera coms{camera=Front Yard id=front_yard}: hik_sink::hikapi::camera: Camera connection established
DEBUG hik_sink::mqtt::connection: Camera event id="front_yard" event=Connected { info: DeviceInfo { device_name: "Front Yard", device_id: "XXXXXX", model: "DS-2CD2342WD-I", serial_number: "DS-2CD2342WD-XXXX", mac_address: "c4:2f:90:XXXXX", firmware_version: "V5.4.5", firmware_release_date: "build 170124", device_type: "IPCamera" }, triggers: [TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: Motion }, hik_id: "VMD-1", description: "VMD Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: Tamper }, hik_id: "tamper-1", description: "shelteralarm Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: DiskFull }, hik_id: "diskfull", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: DiskError }, hik_id: "diskerror", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: NicBroken }, hik_id: "nicbroken", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: IpConflict }, hik_id: "ipconflict", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: IllegalAccess }, hik_id: "illaccess", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: LineDetection }, hik_id: "linedetection-1", description: "Linedetection Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: FieldDetection }, hik_id: "fielddetection-1", description: "fielddetection Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoMismatch }, hik_id: "videomismatch", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: BadVideo }, hik_id: "badvideo", description: "exception Information" }] }
DEBUG hyper::proto::h1::io: parsed 3 headers
DEBUG hyper::proto::h1::conn: incoming body is close-delimited
 INFO Camera coms{camera=Driveway id=driveway}: hik_sink::hikapi::camera: Camera connection established
DEBUG hik_sink::mqtt::connection: Camera event id="front_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:11-8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="driveway" event=Connected { info: DeviceInfo { device_name: "Driveway", device_id: "XXXXXX", model: "DS-2CD2335FWD-I", serial_number: "DS-2CD2335FWD-XXXX", mac_address: "18:68:cb:XXXX", firmware_version: "V5.5.61", firmware_release_date: "build 180718", device_type: "IPCamera" }, triggers: [TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: Motion }, hik_id: "VMD-1", description: "VMD Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: Tamper }, hik_id: "tamper-1", description: "shelteralarm Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: DiskFull }, hik_id: "diskfull", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: DiskError }, hik_id: "diskerror", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: NicBroken }, hik_id: "nicbroken", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: IpConflict }, hik_id: "ipconflict", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: IllegalAccess }, hik_id: "illaccess", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: LineDetection }, hik_id: "linedetection-1", description: "Linedetection Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: FieldDetection }, hik_id: "fielddetection-1", description: "fielddetection Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoMismatch }, hik_id: "videomismatch", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: BadVideo }, hik_id: "badvideo", description: "exception Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: FaceDetection }, hik_id: "facedetection-1", description: "facedetection Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: UnattendedBaggage }, hik_id: "unattendedBaggage-1", description: "UnattendedBaggage Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: AttendedBaggage }, hik_id: "attendedBaggage-1", description: "AttendedBaggage Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: StorageDetection }, hik_id: "storageDetection-1", description: "storageDetection Event trigger Information" }, TriggerItem { identifier: EventIdentifier { channel: Some("1"), event_type: SceneChangeDetection }, hik_id: "scenechangedetection-1", description: "scenechangedetection Event trigger Information" }] }
DEBUG hyper::proto::h1::io: parsed 5 headers
DEBUG hyper::proto::h1::conn: incoming body is content-length (21199 bytes)
DEBUG hyper::proto::h1::conn: incoming body completed
DEBUG Camera coms{camera=North Yard id=north_yard}: hyper::client::connect::http: connecting to 10.1.10.22:80
DEBUG Camera coms{camera=North Yard id=north_yard}: hyper::client::connect::http: connected to 10.1.10.22:80
DEBUG hyper::proto::h1::io: flushed 85 bytes
DEBUG hyper::proto::h1::io: parsed 7 headers
DEBUG hyper::proto::h1::conn: incoming body is content-length (283 bytes)
DEBUG hyper::proto::h1::conn: incoming body completed
DEBUG Camera coms{camera=North Yard id=north_yard}: hyper::client::pool: pooling idle connection for ("http", 10.1.10.22)
DEBUG Camera coms{camera=North Yard id=north_yard}: hyper::client::pool: reuse idle connection for ("http", 10.1.10.22)
DEBUG hyper::proto::h1::io: flushed 373 bytes
DEBUG hyper::proto::h1::io: parsed 2 headers
DEBUG hyper::proto::h1::conn: incoming body is close-delimited
DEBUG hik_sink::mqtt::connection: Camera event id="front_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:17-8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="north_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:16--8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="front_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:17-8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="north_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:17--8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="front_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:17-8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="north_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:17--8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="front_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:17-8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="north_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:17--8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="front_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:18-8:00" })
DEBUG hik_sink::mqtt::connection: Camera event id="north_yard" event=Alert(AlertItem { identifier: EventIdentifier { channel: Some("1"), event_type: VideoLoss }, active: false, regions: [], post_count: 0, description: "videoloss alarm", date: "2021-09-28T15:58:18--8:00" })

CornerBit · September 29, 2021, 2:37pm

That’s very strange! That likely means the camera is able to be queried but the event stream is not. I’d greatly appreciate it if you could send me the rest of the log either here or on Github.

jshank · September 29, 2021, 6:05pm

I solved it by allowing home assistant (mqtt user hass) to read the hikvision_cameras/# topic by modifying my mosquitto.acl. I had only allowed the MQTT hiksink user to read the hikvision_cameras/# topic not realizing that Home Assistant was also referencing that topic for status. (Yes, I have per-topic security via the Mosquitto ACL system)

Other 2mqtt tools (Shellies Discovery and Hass Workstation) feed all of the entity data into the homeassistant/ topic. Maybe an idea for future improvements?

Amazing what you discover when you are preparing the logs to share.

HeyImAlex · September 29, 2021, 6:26pm

Nice work, but what is the advantage of using your bridge over the native Hikvision integration ? I have been using the latter for a long time now, without any issues. I like the MQTT aspect of yours, making it less tied to HA, that’s certainly a plus. Any other unique selling point of yours ?

CornerBit · September 30, 2021, 5:10am

Glad to hear you’ve solved it! That’d definitely something to consider for a future version. I based the current split based on the way ESPHome and Zigbee2MQTT do it to make it easier to integrate with non-homeassistant software.

CornerBit · September 30, 2021, 5:15am

Thanks! If the current Hikvision integration works for you, you probably won’t see too much benefit from switching. This fixes a some of the reliability quirks I had with HA where cameras would silently stop producing events after temporary network issues. It’s also able to discover new alert types which gave me access to more triggers (e.g. Illegal logins). Although as you mentioned, the main reason to use HikSink would be for it’s HA-independence so you can process data in NodeRED or in your own scripts

rindlerblabla · September 30, 2021, 12:10pm

@CornerBit Do you think it would be possible in some way to turn on the camera LED on motion? Would be nice if the LED would appear as a switch in the UI.

CornerBit · September 30, 2021, 12:39pm

Unfortunately I’m not aware of any way to directly control the camera’s LED - it may be possible, but I haven’t seen come across it in the API. An alternate (albeit less elegant) way would be to get a smart flood light and automate that over Home Assistant

bigbigblue · June 29, 2022, 11:01pm

Sorry for the numpty question, but is there a way of installing this on HassOS?
I can’t install rust compiler as ‘apt’ command is not found in Home Assistant SSH add-on terminal.
Thanks

CornerBit · July 5, 2022, 1:24pm

I’m not familiar with HassOS, but it should be doable. You can install rust via https://rustup.rs/ . The build command should then work. My only concern would be missing system libraries, but there’s a good chance they are shipped with HassOS.

bigbigblue · July 8, 2022, 1:52pm

Unfortunately, the curl command at https://rustup.rs returns the following when attempting to run on raspberry pi via the Home Assistant Terminal add-on

info: downloading installer
curl (22): The requested URL returned error: 404
rustup: installer for platform ‘armv7-unknown-linux-musleabihf’ not found, this might be unsupported

bigbigblue · August 5, 2022, 1:25pm

I have tried to compile this on my Raspberry Pi (as there is no armv7 version of the container) and I receive the following errors:

error: failed to parse manifest at /hiksink/Hiksink_code/Cargo.toml

Caused by:
failed to parse the edition key

Caused by:
supported edition values are 2015 or 2018, but 2021 is unknown
xxxxxxxx@raspberrypi:/hiksink/Hiksink_code $ nano Cargo.toml
xxxxxxxx@raspberrypi:/hiksink/Hiksink_code $ cargo build --release
Updating crates.io index
error: failed to get chrono as a dependency of package hik_sink v1.2.1 (/hiksink/Hiksink_code)

Caused by:
failed to fetch https://github.com/rust-lang/crates.io-index

Caused by:
network failure seems to have happened
if a proxy or similar is necessary net.git-fetch-with-cli may help here
Configuration - The Cargo Book

Caused by:
SSL error: 0xffff8880 - SSL - A fatal alert message was received from our peer; class=Ssl (16)

CornerBit · August 13, 2022, 1:43am

It looks like there may have been network config issues on your raspberry pi, or possibly a certificate based issue. Are you able to access other secure sites, e.g. https://www.rust-lang.org/ via curl?

bigbigblue · August 13, 2022, 9:42am

Thanks for the reply. I installed Debian (rather than Rasperry Pi OS) and managed to compile and run HikSink
I was hoping that HikSink would report smart events (specifically intrusion, zone entrance and zone exit ) separately to motion, but there were not. I presume this is due to the Hikvision api not reporting them separately?
It would really help my automations If I could separate the smart events from simple motion detection.
I suppose I could set the ‘normal’ motion detection in the camera to not report to security centre, then any ‘motion’ detection would be one of the smart events. I also think that could be done with the standard Home Assistant Hikvision integration.
Any thoughts on the above?
Cheers