Notification with cropped image from camera - Node Red

I’m trying to get a cropped image from my driveway camera when the door sensor on the mailbox opens. I don’t care if it is in node-red (I am super new to NR) but I don’t know how to do this in the automation UI in HA either.

In NR, I have been able to send a pic notification to an iPhone with a snapshot of the camera but it uses a local URL so it doesn’t work outside of home (5G cellular, etc.). Problem 1: Do I have to use an external URL to view a pic on a notification or can I pass the image as part of the notification?

Problem 2: I am using the Image node but keep running into an error: “Error: No matching constructor overloading was found. Please see the docs for how to call the Jimp constructor.” I have tried to pass the image from a msg.payload from a Call Service node which capture the snapshot from the camera. I have tried searching for an answer but haven’t been able to find it at least for node-red. This may be a really stupid question but do I need to install jimp?

Thank you!
Mike

Neither. Use the a media source link as described here. Then it works locally or externally and you don’t have to worry about it.

Normally I direct people to the automatic snapshot option as its even easier but since you have to crop the image first in this case that won’t work for you. But providing a direct internal or external URL is the third option in that documentation which is the one specifically not recommended. Since its the only one that requires you put the image somewhere publicly accessible without authentication. All the other options handle on and off LAN without issue and still require authenticated access to the image.

Can’t really help you with cropping much as I’ve never tried anything like that. If you know how to do what you’re trying to do on the command line though could try just making a shell command.

Is the crop dynamic? Meaning are you trying to isolate something moving in the picture? That’s not easy if it is possible. If it’s a simple crop image tools can handle it.

If it is dynamic check out AI photo booth. I haven’t used or read through it yet but it may be able to do a dynamic crop.

Digging into this now. Thank you!

Static, fortunately. I have a 4k cam over the driveway that I wanted to crop since the mailbox is always in the same place. The idea is I get a cropped pic of whoever opened the mailbox. Cam is integrated with Onvif. I found the Camera Proxy integration, after I started this process, so I have not attempted it yet.

I should have mentioned that I was trying to use the image tools you listed. I have attached the flow I am attempting. I think I am missing something with the picture passing into the image tools. This node is where the error “Error: No matching constructor overloading was found. Please see the docs for how to call the Jimp constructor.” seems to be coming from. I will keep trying. Thank you!

if you’re using the camera.snapshot service call it saves the snap shot to a file that you would then need to read into NR using a file in node.

There is a good discussion about Node Red and Image processing over on the Node Red Discourse forum

Craig

Sorry all, had a family emergency and haven’t had a chance to work on this. Today I am able to get the picture to save in the “local” directory in media/local/mailbox_snapshot. I can open the file and view it. I keep getting the error: Error: No matching constructor overloading was found. Please see the docs for how to call the Jimp constructor. I have searched this and either can’t find the answer or don’t know what to look for. Is there a file/program that I need to install on HA? I’m using the HA OS on a rPi 4. I’ve tried npm install --save jimp in the terminal but it doesn’t work.

Last thing: Is this the right way to reference the image locally in the image node?:
{“filename”:"/media/local/mailbox_snapshot.jpg"}

Thank you!

Also, here is the flow:

[{“id”:“ea0f41b8203694ad”,“type”:“tab”,“label”:“Mailbox Notification”,“disabled”:false,“info”:"",“env”:[]},{“id”:“12958436e6314449”,“type”:“trigger-state”,“z”:“ea0f41b8203694ad”,“name”:“Mailbox Sensor Opened”,“server”:“6119efaf.04ac5”,“version”:2,“exposeToHomeAssistant”:false,“haConfig”:[{“property”:“name”,“value”:""},{“property”:“icon”,“value”:""}],“entityid”:“binary_sensor.mailbox_contact_sensor”,“entityidfiltertype”:“exact”,“debugenabled”:false,“constraints”:[{“targetType”:“this_entity”,“targetValue”:"",“propertyType”:“current_state”,“propertyValue”:“new_state.state”,“comparatorType”:“is”,“comparatorValueDatatype”:“str”,“comparatorValue”:“on”}],“inputs”:0,“outputs”:2,“customoutputs”:[],“outputinitially”:true,“state_type”:“str”,“enableInput”:false,“x”:130,“y”:220,“wires”:[[“f55442d00fb29faf”],[]]},{“id”:“f55442d00fb29faf”,“type”:“api-call-service”,“z”:“ea0f41b8203694ad”,“name”:“Driveway Snapshot”,“server”:“6119efaf.04ac5”,“version”:5,“debugenabled”:false,“domain”:“camera”,“service”:“snapshot”,“areaId”:[],“deviceId”:[],“entityId”:[“camera.driveway_mainstream”],“data”:"{“filename”:"/media/local/mailbox_snapshot.jpg"}",“dataType”:“json”,“mergeContext”:"",“mustacheAltTags”:false,“outputProperties”:[{“property”:“payload”,“propertyType”:“msg”,“value”:“image”,“valueType”:“flow”}],“queue”:“none”,“x”:350,“y”:180,“wires”:[[“2f041f3f90268f90”,“7a3a969c0601546b”,“6c42819836f31c7f”]]},{“id”:“50671b23966a2f2b”,“type”:“inject”,“z”:“ea0f41b8203694ad”,“name”:"",“props”:[{“p”:“payload”},{“p”:“topic”,“vt”:“str”}],“repeat”:"",“crontab”:"",“once”:false,“onceDelay”:0.1,“topic”:"",“payload”:"",“payloadType”:“date”,“x”:160,“y”:160,“wires”:[[“f55442d00fb29faf”]]},{“id”:“2f041f3f90268f90”,“type”:“jimp-image”,“z”:“ea0f41b8203694ad”,“name”:“Picture Crop”,“data”:"{“filename”:"/media/local/mailbox_snapshot.jpg"}",“dataType”:“json”,“ret”:“b64”,“parameter1”:“50”,“parameter1Type”:“num”,“parameter2”:“50”,“parameter2Type”:“num”,“parameter3”:“100”,“parameter3Type”:“num”,“parameter4”:“100”,“parameter4Type”:“num”,“parameter5”:"",“parameter5Type”:“msg”,“parameter6”:"",“parameter6Type”:“msg”,“parameter7”:"",“parameter7Type”:“msg”,“parameter8”:"",“parameter8Type”:“msg”,“sendProperty”:“payload”,“sendPropertyType”:“msg”,“parameterCount”:4,“jimpFunction”:“crop”,“selectedJimpFunction”:{“name”:“crop”,“fn”:“crop”,“description”:“crop to the given region”,“parameters”:[{“name”:“x”,“type”:“num”,“required”:true,“hint”:“the x coordinate to crop form”},{“name”:“y”,“type”:“num”,“required”:true,“hint”:“the y coordinate to crop form”},{“name”:“w”,“type”:“num”,“required”:true,“hint”:“the width of the crop region”},{“name”:“h”,“type”:“num”,“required”:true,“hint”:“the height of the crop region”}]},“x”:590,“y”:180,“wires”:[[“7d6d31443862c01a”,“19fe979485d6a79b”]]},{“id”:“7d6d31443862c01a”,“type”:“api-call-service”,“z”:“ea0f41b8203694ad”,“name”:“Notify Mike”,“server”:“6119efaf.04ac5”,“version”:5,“debugenabled”:false,“domain”:“notify”,“service”:“mobile_app_nacho”,“areaId”:[],“deviceId”:[],“entityId”:[],“data”:"{“title”:“Mailbox Opened”,“message”:“Mailbox Opened”,“data”:{“attachment”:{“content_type”:“jpeg”,“uri”:“media-source://media_source/local/mailbox_snapshot.jpg”}}}",“dataType”:“json”,“mergeContext”:"",“mustacheAltTags”:false,“outputProperties”:[],“queue”:“none”,“x”:770,“y”:180,“wires”:[[]]},{“id”:“19fe979485d6a79b”,“type”:“image viewer”,“z”:“ea0f41b8203694ad”,“name”:"",“width”:160,“data”:“payload”,“dataType”:“msg”,“active”:true,“x”:770,“y”:240,“wires”:[[]]},{“id”:“25f256114d171191”,“type”:“api-call-service”,“z”:“ea0f41b8203694ad”,“name”:“Notify Mike”,“server”:“6119efaf.04ac5”,“version”:5,“debugenabled”:false,“domain”:“notify”,“service”:“mobile_app_nacho”,“areaId”:[],“deviceId”:[],“entityId”:[],“data”:"{“title”:“Mailbox Opened”,“message”:“Mailbox Opened”,“data”:{“attachment”:{“content_type”:“jpeg”,“uri”:“media-source://media_source/local/mailbox_snapshot.jpg”}}}",“dataType”:“json”,“mergeContext”:"",“mustacheAltTags”:false,“outputProperties”:[],“queue”:“none”,“x”:710,“y”:60,“wires”:[[]]},{“id”:“6c42819836f31c7f”,“type”:“debug”,“z”:“ea0f41b8203694ad”,“name”:“debug 1”,“active”:true,“tosidebar”:true,“console”:false,“tostatus”:false,“complete”:“true”,“targetType”:“full”,“statusVal”:"",“statusType”:“auto”,“x”:660,“y”:320,“wires”:[]},{“id”:“7a3a969c0601546b”,“type”:“delay”,“z”:“ea0f41b8203694ad”,“name”:"",“pauseType”:“delay”,“timeout”:“1”,“timeoutUnits”:“seconds”,“rate”:“1”,“nbRateUnits”:“1”,“rateUnits”:“second”,“randomFirst”:“1”,“randomLast”:“5”,“randomUnits”:“seconds”,“drop”:false,“allowrate”:false,“outputs”:1,“x”:540,“y”:60,“wires”:[[“25f256114d171191”]]},{“id”:“6119efaf.04ac5”,“type”:“server”,“name”:“Home Assistant”,“version”:4,“addon”:true,“rejectUnauthorizedCerts”:true,“ha_boolean”:“y|yes|true|on|home|open”,“connectionDelay”:true,“cacheJson”:true,“heartbeat”:false,“heartbeatInterval”:30,“areaSelector”:“friendlyName”,“deviceSelector”:“friendlyName”,“entitySelector”:“friendlyName”,“statusSeparator”:"at: ",“statusYear”:“hidden”,“statusMonth”:“short”,“statusDay”:“numeric”,“statusHourCycle”:“h23”,“statusTimeFormat”:“h:m”}]

I have finally got this to work, now tweaking the cropping parameters. Thank you Kermit! I will attach the code for the flow shortly for future inquiries to this thread.

Here is the code for the flow:

[{“id”:“ea0f41b8203694ad”,“type”:“tab”,“label”:“Mailbox Notification”,“disabled”:false,“info”:"",“env”:[]},{“id”:“12958436e6314449”,“type”:“trigger-state”,“z”:“ea0f41b8203694ad”,“name”:“Mailbox Sensor Opened”,“server”:“6119efaf.04ac5”,“version”:2,“exposeToHomeAssistant”:false,“haConfig”:[{“property”:“name”,“value”:""},{“property”:“icon”,“value”:""}],“entityid”:“binary_sensor.mailbox_contact_sensor”,“entityidfiltertype”:“exact”,“debugenabled”:false,“constraints”:[{“targetType”:“this_entity”,“targetValue”:"",“propertyType”:“current_state”,“propertyValue”:“new_state.state”,“comparatorType”:“is”,“comparatorValueDatatype”:“str”,“comparatorValue”:“on”}],“inputs”:0,“outputs”:2,“customoutputs”:[],“outputinitially”:true,“state_type”:“str”,“enableInput”:false,“x”:130,“y”:160,“wires”:[[“f55442d00fb29faf”],[]]},{“id”:“f55442d00fb29faf”,“type”:“api-call-service”,“z”:“ea0f41b8203694ad”,“name”:“Driveway Snapshot”,“server”:“6119efaf.04ac5”,“version”:5,“debugenabled”:false,“domain”:“camera”,“service”:“snapshot”,“areaId”:[],“deviceId”:[],“entityId”:[“camera.driveway_mainstream”],“data”:"{“filename”:"/media/local/mailbox_snapshot.jpg"}",“dataType”:“json”,“mergeContext”:"",“mustacheAltTags”:false,“outputProperties”:[{“property”:“payload”,“propertyType”:“msg”,“value”:“image”,“valueType”:“flow”}],“queue”:“none”,“x”:390,“y”:160,“wires”:[[“fd5cfd8786be2985”]]},{“id”:“50671b23966a2f2b”,“type”:“inject”,“z”:“ea0f41b8203694ad”,“name”:"",“props”:[{“p”:“payload”},{“p”:“topic”,“vt”:“str”}],“repeat”:"",“crontab”:"",“once”:false,“onceDelay”:0.1,“topic”:"",“payload”:"",“payloadType”:“date”,“x”:140,“y”:100,“wires”:[[“f55442d00fb29faf”]]},{“id”:“7d6d31443862c01a”,“type”:“api-call-service”,“z”:“ea0f41b8203694ad”,“name”:“Notify Mike”,“server”:“6119efaf.04ac5”,“version”:5,“debugenabled”:false,“domain”:“notify”,“service”:“mobile_app_nacho”,“areaId”:[],“deviceId”:[],“entityId”:[],“data”:"{“title”:“Mailbox Opened”,“message”:“Mailbox Opened”,“data”:{“attachment”:{“content_type”:“jpeg”,“url”:"/media/local/local/mailbox_snapshot_cropped.jpg"}}}",“dataType”:“json”,“mergeContext”:"",“mustacheAltTags”:false,“outputProperties”:[],“queue”:“none”,“x”:670,“y”:280,“wires”:[[]]},{“id”:“fd5cfd8786be2985”,“type”:“file in”,“z”:“ea0f41b8203694ad”,“name”:"",“filename”:"/media/local/mailbox_snapshot.jpg",“filenameType”:“str”,“format”:"",“chunk”:false,“sendError”:false,“encoding”:“base64”,“allProps”:false,“x”:680,“y”:160,“wires”:[[“87b0d963d1bbab6c”]]},{“id”:“87b0d963d1bbab6c”,“type”:“jimp-image”,“z”:“ea0f41b8203694ad”,“name”:"",“data”:“payload”,“dataType”:“msg”,“ret”:“buf”,“parameter1”:“425”,“parameter1Type”:“num”,“parameter2”:“0”,“parameter2Type”:“num”,“parameter3”:“1500”,“parameter3Type”:“num”,“parameter4”:“1200”,“parameter4Type”:“num”,“parameter5”:"",“parameter5Type”:“msg”,“parameter6”:"",“parameter6Type”:“msg”,“parameter7”:"",“parameter7Type”:“msg”,“parameter8”:"",“parameter8Type”:“msg”,“sendProperty”:“payload”,“sendPropertyType”:“msg”,“parameterCount”:4,“jimpFunction”:“crop”,“selectedJimpFunction”:{“name”:“crop”,“fn”:“crop”,“description”:“crop to the given region”,“parameters”:[{“name”:“x”,“type”:“num”,“required”:true,“hint”:“the x coordinate to crop form”},{“name”:“y”,“type”:“num”,“required”:true,“hint”:“the y coordinate to crop form”},{“name”:“w”,“type”:“num”,“required”:true,“hint”:“the width of the crop region”},{“name”:“h”,“type”:“num”,“required”:true,“hint”:“the height of the crop region”}]},“x”:110,“y”:280,“wires”:[[“82e4c665a5c59440”]]},{“id”:“82e4c665a5c59440”,“type”:“file”,“z”:“ea0f41b8203694ad”,“name”:"",“filename”:"/media/local/mailbox_snapshot_cropped.jpg",“filenameType”:“str”,“appendNewline”:false,“createDir”:false,“overwriteFile”:“true”,“encoding”:“none”,“x”:390,“y”:280,“wires”:[[“7d6d31443862c01a”]]},{“id”:“6119efaf.04ac5”,“type”:“server”,“name”:“Home Assistant”,“version”:4,“addon”:true,“rejectUnauthorizedCerts”:true,“ha_boolean”:“y|yes|true|on|home|open”,“connectionDelay”:true,“cacheJson”:true,“heartbeat”:false,“heartbeatInterval”:30,“areaSelector”:“friendlyName”,“deviceSelector”:“friendlyName”,“entitySelector”:“friendlyName”,“statusSeparator”:"at: ",“statusYear”:“hidden”,“statusMonth”:“short”,“statusDay”:“numeric”,“statusHourCycle”:“h23”,“statusTimeFormat”:“h:m”}]

2 Likes

I’m glad you solved this since suddenly I actually have a need for it as well lol.

For future reference, when sharing JSON please wrap it in triple backticks (```). Otherwise it gets processed as markdown which does weird things. Like make all your quotation marks into smart quotes so it can’t be copied and pasted into node red import (which is what happened above).

Since I fixed it so I could import it I figured it I’d reshare without the smart quotes

EDIT 2: Removed my attempt to fix it because its still broken. Didn’t want to give folks the wrong idea.

EDIT: Ok it apparently added some random quotes too? Not sure how that happened. Anyway fixed that too, now its importable.

That’s how it’s done! Thank you. I was hoping this might help someone else, the community here is amazing!

Oh wait I screwed it up. I see what happened, when there’s json as the value of the field things got really weird. It replaced the quotes inside with smart quotes but the ones surrounding it were left alone. So when I stripped the outer ones out to make it valid JSON I ended up with this on import:

Which isn’t correct. If you don’t mind could you re-export and put it in the backticks so its easily importable?

Let’s see if this works:

[{"id":"ea0f41b8203694ad","type":"tab","label":"Mailbox Notification","disabled":false,"info":"","env":[]},{"id":"1e84c7e404c0af12","type":"trigger-state","z":"ea0f41b8203694ad","name":"Mailbox Sensor Opened","server":"6119efaf.04ac5","version":2,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityid":"binary_sensor.mailbox_contact_sensor","entityidfiltertype":"exact","debugenabled":false,"constraints":[{"targetType":"this_entity","targetValue":"","propertyType":"current_state","propertyValue":"new_state.state","comparatorType":"is","comparatorValueDatatype":"str","comparatorValue":"on"}],"inputs":0,"outputs":2,"customoutputs":[],"outputinitially":true,"state_type":"str","enableInput":false,"x":130,"y":80,"wires":[["f71d21a60efc25f8"],[]]},{"id":"f71d21a60efc25f8","type":"api-call-service","z":"ea0f41b8203694ad","name":"Driveway Snapshot","server":"6119efaf.04ac5","version":5,"debugenabled":false,"domain":"camera","service":"snapshot","areaId":[],"deviceId":[],"entityId":["camera.driveway_mainstream"],"data":"{\"filename\":\"/media/snapshots/mailbox_snapshot.jpg\"}","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"image","valueType":"flow"}],"queue":"none","x":390,"y":80,"wires":[["6bcd77a41629f9a7"]]},{"id":"31c0aca40b4827e1","type":"inject","z":"ea0f41b8203694ad","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":100,"y":40,"wires":[["f71d21a60efc25f8"]]},{"id":"f61b371f1fad31a7","type":"api-call-service","z":"ea0f41b8203694ad","name":"Notify All","server":"6119efaf.04ac5","version":5,"debugenabled":false,"domain":"notify","service":"notify","areaId":[],"deviceId":[],"entityId":[],"data":"{\"title\":\"Mailbox\",\"message\":\"Mailbox Opened\",\"data\":{\"attachment\":{\"content_type\":\"jpeg\",\"url\":\"/media/local/snapshots/mailbox_snapshot_cropped.jpg\"}}}","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":700,"y":200,"wires":[[]]},{"id":"6bcd77a41629f9a7","type":"file in","z":"ea0f41b8203694ad","name":"","filename":"/media/snapshots/mailbox_snapshot.jpg","filenameType":"str","format":"","chunk":false,"sendError":false,"encoding":"base64","allProps":false,"x":700,"y":80,"wires":[["fafacd80469b5e62"]]},{"id":"fafacd80469b5e62","type":"jimp-image","z":"ea0f41b8203694ad","name":"","data":"payload","dataType":"msg","ret":"buf","parameter1":"425","parameter1Type":"num","parameter2":"0","parameter2Type":"num","parameter3":"1500","parameter3Type":"num","parameter4":"1200","parameter4Type":"num","parameter5":"","parameter5Type":"msg","parameter6":"","parameter6Type":"msg","parameter7":"","parameter7Type":"msg","parameter8":"","parameter8Type":"msg","sendProperty":"payload","sendPropertyType":"msg","parameterCount":4,"jimpFunction":"crop","selectedJimpFunction":{"name":"crop","fn":"crop","description":"crop to the given region","parameters":[{"name":"x","type":"num","required":true,"hint":"the x coordinate to crop form"},{"name":"y","type":"num","required":true,"hint":"the y coordinate to crop form"},{"name":"w","type":"num","required":true,"hint":"the width of the crop region"},{"name":"h","type":"num","required":true,"hint":"the height of the crop region"}]},"x":110,"y":200,"wires":[["57b93fefd54c8c7b"]]},{"id":"57b93fefd54c8c7b","type":"file","z":"ea0f41b8203694ad","name":"","filename":"/media/snapshots/mailbox_snapshot_cropped.jpg","filenameType":"str","appendNewline":false,"createDir":false,"overwriteFile":"true","encoding":"none","x":410,"y":200,"wires":[["f61b371f1fad31a7"]]},{"id":"6119efaf.04ac5","type":"server","name":"Home Assistant","version":4,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":30,"areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m"}]

2 Likes

Definitely better, is copy and pasteable. Although for reference, when you have a large block of code like that it’s usually better to surround it in triple backticks instead of just one. Then it makes a nice block view with a copy button like this:

This is text surrounded with triple backticks (```) before and after

Also for future readers, I figured out a way to do this without using Node RED. Although I did need another addon instead since I needed to install imagemagick somewhere. You’ll need the ssh addon from the community repo and then add imagemagick to the list of installed apks in its config. Also you’ll need to set up the ability to ssh into the SSH addon from a shell command, see this guide if you haven’t done that before:

Once you have that set up then you can use this command line notify to crop an image before sending it in a notification or whatever:

notify:
- platform: command_line
  name: Crop image
  command: >-
    cat > /config/.crop_args
    && crop_cmd='args=(${(s/,/)$(cat /config/.crop_args)}) && convert "${args[1]}" -crop "${args[2]}" /media/.tmp_crop 2> /dev/null && mv /media/.tmp_crop "${args[1]}"'
    && ssh -o UserKnownHostsFile=/config/.ssh/known_hosts root@localhost -i /config/.ssh/id_ed25519 "${crop_cmd}"
    && rm /config/.crop_args

Note that this is relies on zsh being the shell within the addon, that’s why the SSH addon from the community repo specifically is required. It could be adjusted to use bash and then would work with the SSH addon in the official repo. You also may need to adjust /config/.ssh/known_hosts and /config/.ssh/id_ed25519 to whatever files you used for those from the guide above.

You can then use it like this:

- alias: Crop an image
  service: notify.crop_image
  data:
    message: "/media/snapshot.jpg,1520x1344+0+1344" # <image>,<Width>x<Height>+<x>+<y>

It’ll crop the image listed to the dimensions specified and then replace the original file with it.

Thanks for posting this I hadn’t realised you could crop with the image node.