Curl request with state sensor and parse ouput of that request

Third party integrations Node-RED
1

hello!

I’m currently trying to do this in HA but it does not seem to support this (short of an integration).

Here’s what I like to do. I have another machine that does AI including image generation based on prompts. It works via curl request. I originally had it set up as a rest sensor in Home Asissistant (see below), that sends the request to the LocalAI machine, and returns a value for the sensor, being where the image URL is located. I then point an image card to that sensor and it shows the picture on my HA dashboard BUT I later learned that it was not actually passing through the template sensor values. For example, below in the “prompt” field, there are few sensors that show this template {{ states('sensor.humidex_perception') }}, which is basically a string value for the sensor state. This should become part of the prompt as a string when it reaches the LocalAI machine

#LocalAI Image Generations
- platform: rest
  name: "Image URL Sensor"
  unique_id: [id]
  resource: http://192.168.1.195:8080/v1/images/generations
  method: POST
  scan_interval: 43200
  timeout: 120 
  headers:
    Content-Type: application/json
  payload: > 
    {
    "prompt": "magnificent scenery, wide landscape, sharp and crisp background, very beautiful landscape, fantasy, birdview, best quality, masterpiece, ultra high res, dark blue light, photo, photorealistic, wide view, kkw-ph1, professional shot of house, {{ state_attr('weather.forecast_home', 'forecast')[0].condition }} day, humidity perception is {{ states('sensor.humidex_perception') }}, thermal perception is {{ states('sensor.thermalperception') }}, extreme low-angle shot, realistic photography, ultra detailedmagnificent scenery, wide landscape, sharp and crisp background, very beautiful landscape, old ruins buildings, fantasy, birdview, best quality, masterpiece, ultra high res, dark blue light, cloudy, photo, photorealistic, wide viewl",
    "size": "512x512",
    "model": "landscape-photoreal"
    }
  value_template: "{{ value_json.data[0].url }}"

it was passing all those template sensors as templates.

Is this something that I can solve in NodeRed as im not sure if the prompt will actually have the state sensor in it (as string).

thank you!

2

Yes you can do it in Node Red. The reason it’s not working as a REST sensor is because payload (currently) only supports a string, not a template. In NR you would just use an HTTP request node, and pass the payload into it. That payload can be created from whatever source of data - in your case, probably current-state nodes retrieving the forecast, humidity, etc.

1 Like
3

thank you, @michaelblight

I’ve created a simple flow here

image

where the inject node is

{
    "prompt": "A vast cityscape stretches out before you, its towering skyscrapers piercing the clouds. Neon lights illuminate streets, creating a vibrant and dynamic atmosphere. Pedestrians hurry along the sidewalks. The city is alive with energy, the weather is {{ state_attr('weather.forecast_home', 'forecast')[0].condition }} ",
    "size": "512x512",
    "model": "landscape-photoreal"
}

Using the current state node, I don’t see the option to add the extra lines i.e. prompt. size, model etc.

Or do I ened render template node and dump all this?

thanks for your help

4

I’m not a big fan of the template node as it hides the complexity inside, and I prefer to be able to see all the steps going on in a flow. So I would have 3 current state nodes, getting forecast, humidity and thermal perception from HA. For the second two, you’re using the state, so that will be returned in payload, however, you don’t want one payload overwriting the next, so you can tell it to put the state in (for example) payload_humidex and payload_thermal. For the forecast, you’re using an attribute, so this is in data rather than payload, so that’s fine.

Then you could have a change node that sets the payload for the http node using JSONata along the lines of:

{
    "prompt":
        "magnificent scenery, wide landscape, sharp etc, " &
        data.attributes.forecast[0].condition &
        " day, humidity perception is " &
        payload_humidex &
        ", thermal perception is " &
        payload_thermal &
        ", extreme low-angle etc",
    "size": "512x512",
    "model": "landscape-photoreal"
}

All of the nodes should just be joined together one after the other - i.e. none in parallel. It’s possible I’ve misunderstood the path for the forecast attribute - you can stick a debug node after it, outputting the complete msg object, then run and drill down to what you’re after and click the “copy path” icon.

The way you’re current code is set up is using mustache templates, which are different to HA templates, and only exist in NR’s template node and the HA nodes. So it won’t work in an inject node or change node.

And note there are many other ways of doing the same thing, but this is the simplest (IMHO).

1 Like
5

thanks very much!

Problem solved! This is very similar to what you’re saying below:

image
image1293×320 61 KB

I have to figure out how to trigger this in HA though. Clearly this is triggered with a time stamp manually, maybe some trigger node?

EDIT: nevermind, created a button which is used in an automation to trigger the flow

6

No worries. You don’t really need the 3 set topics and join - you can just link the current state nodes together, as long as you change their settings so they don’t override each other.

7

you’re right, that would reduce the number of nodes. I may considered modying this:

Json flow 
[
    {
        "id": "2f2bf929e62df7e3",
        "type": "tab",
        "label": "LocalAI Weather Image",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "531a8ede1ea6f42e",
        "type": "api-current-state",
        "z": "2f2bf929e62df7e3",
        "name": "Weather Forcast",
        "server": "505255aec0a0bb5e",
        "version": 3,
        "outputs": 1,
        "halt_if": "",
        "halt_if_type": "str",
        "halt_if_compare": "is",
        "entity_id": "weather.forecast_home",
        "state_type": "str",
        "blockInputOverrides": false,
        "outputProperties": [
            {
                "property": "payload",
                "propertyType": "msg",
                "value": "",
                "valueType": "entityState"
            },
            {
                "property": "data",
                "propertyType": "msg",
                "value": "",
                "valueType": "entity"
            }
        ],
        "for": "0",
        "forType": "num",
        "forUnits": "minutes",
        "override_topic": false,
        "state_location": "payload",
        "override_payload": "msg",
        "entity_location": "data",
        "override_data": "msg",
        "x": 440,
        "y": 100,
        "wires": [
            [
                "d4ec411aac0d5bfc"
            ]
        ]
    },
    {
        "id": "c885b3997b7a27e5",
        "type": "api-current-state",
        "z": "2f2bf929e62df7e3",
        "name": "Humidex Perception",
        "server": "505255aec0a0bb5e",
        "version": 3,
        "outputs": 1,
        "halt_if": "",
        "halt_if_type": "str",
        "halt_if_compare": "is",
        "entity_id": "sensor.humidex_perception",
        "state_type": "str",
        "blockInputOverrides": false,
        "outputProperties": [
            {
                "property": "payload",
                "propertyType": "msg",
                "value": "",
                "valueType": "entityState"
            },
            {
                "property": "data",
                "propertyType": "msg",
                "value": "",
                "valueType": "entity"
            }
        ],
        "for": "0",
        "forType": "num",
        "forUnits": "minutes",
        "override_topic": false,
        "state_location": "payload",
        "override_payload": "msg",
        "entity_location": "data",
        "override_data": "msg",
        "x": 460,
        "y": 160,
        "wires": [
            [
                "0db39c3196a3666e"
            ]
        ]
    },
    {
        "id": "46dd8e28bf5da6f9",
        "type": "api-current-state",
        "z": "2f2bf929e62df7e3",
        "name": "Thermal Perception",
        "server": "505255aec0a0bb5e",
        "version": 3,
        "outputs": 1,
        "halt_if": "",
        "halt_if_type": "str",
        "halt_if_compare": "is",
        "entity_id": "sensor.thermalperception",
        "state_type": "str",
        "blockInputOverrides": false,
        "outputProperties": [
            {
                "property": "payload",
                "propertyType": "msg",
                "value": "",
                "valueType": "entityState"
            },
            {
                "property": "data",
                "propertyType": "msg",
                "value": "",
                "valueType": "entity"
            }
        ],
        "for": "0",
        "forType": "num",
        "forUnits": "minutes",
        "override_topic": false,
        "state_location": "payload",
        "override_payload": "msg",
        "entity_location": "data",
        "override_data": "msg",
        "x": 450,
        "y": 220,
        "wires": [
            [
                "95db309414a6b180"
            ]
        ]
    },
    {
        "id": "d4ec411aac0d5bfc",
        "type": "change",
        "z": "2f2bf929e62df7e3",
        "name": "set topic",
        "rules": [
            {
                "t": "set",
                "p": "topic",
                "pt": "msg",
                "to": "name1",
                "tot": "str"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 660,
        "y": 120,
        "wires": [
            [
                "5603fed743446324"
            ]
        ]
    },
    {
        "id": "0db39c3196a3666e",
        "type": "change",
        "z": "2f2bf929e62df7e3",
        "name": "set topic",
        "rules": [
            {
                "t": "set",
                "p": "topic",
                "pt": "msg",
                "to": "name2",
                "tot": "str"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 660,
        "y": 160,
        "wires": [
            [
                "5603fed743446324"
            ]
        ]
    },
    {
        "id": "95db309414a6b180",
        "type": "change",
        "z": "2f2bf929e62df7e3",
        "name": "set topic",
        "rules": [
            {
                "t": "set",
                "p": "topic",
                "pt": "msg",
                "to": "name3",
                "tot": "str"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 660,
        "y": 200,
        "wires": [
            [
                "5603fed743446324"
            ]
        ]
    },
    {
        "id": "5603fed743446324",
        "type": "join",
        "z": "2f2bf929e62df7e3",
        "name": "",
        "mode": "custom",
        "build": "object",
        "property": "payload",
        "propertyType": "msg",
        "key": "topic",
        "joiner": "\\n",
        "joinerType": "str",
        "accumulate": false,
        "timeout": "10",
        "count": "3",
        "reduceRight": false,
        "reduceExp": "",
        "reduceInit": "",
        "reduceInitType": "",
        "reduceFixup": "",
        "x": 790,
        "y": 160,
        "wires": [
            [
                "24ba1bc44d5f8018"
            ]
        ]
    },
    {
        "id": "24ba1bc44d5f8018",
        "type": "template",
        "z": "2f2bf929e62df7e3",
        "name": "",
        "field": "payload",
        "fieldType": "msg",
        "format": "handlebars",
        "syntax": "mustache",
        "template": "{\n    \"prompt\": \"A vast cityscape stretches out before you, its towering skyscrapers piercing the clouds. Neon lights illuminate streets, creating a vibrant and dynamic atmosphere. Pedestrians hurry along the sidewalks. The city is alive with energy, the weather is {{payload.name1}}, {{payload.name2}}, {{payload.name3}} .|nude, human, nudes, naked, sex, horny, sexual\",\n    \"size\": \"512x512\",\n    \"model\": \"landscape-photoreal\"\n}",
        "output": "json",
        "x": 920,
        "y": 220,
        "wires": [
            [
                "1c74eaa1a2d60763"
            ]
        ]
    },
    {
        "id": "1c74eaa1a2d60763",
        "type": "http request",
        "z": "2f2bf929e62df7e3",
        "name": "",
        "method": "POST",
        "ret": "obj",
        "paytoqs": "ignore",
        "url": "http://192.168.1.195:8080/v1/images/generations",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [
            {
                "keyType": "Content-Type",
                "keyValue": "",
                "valueType": "application/json",
                "valueValue": ""
            }
        ],
        "x": 1070,
        "y": 160,
        "wires": [
            [
                "326887438d9475f1"
            ]
        ]
    },
    {
        "id": "326887438d9475f1",
        "type": "ha-sensor",
        "z": "2f2bf929e62df7e3",
        "name": "LocalAI Image URL",
        "entityConfig": "3a531b742507ca90",
        "version": 0,
        "state": "payload.data[0].url",
        "stateType": "msg",
        "attributes": [],
        "inputOverride": "allow",
        "outputProperties": [],
        "x": 1290,
        "y": 260,
        "wires": [
            []
        ]
    },
    {
        "id": "03cb81a0cf2092b3",
        "type": "ha-button",
        "z": "2f2bf929e62df7e3",
        "name": "LocalAI Image Button",
        "version": 0,
        "debugenabled": false,
        "outputs": 1,
        "entityConfig": "c1f022eec4da33cb",
        "outputProperties": [
            {
                "property": "payload",
                "propertyType": "msg",
                "value": "",
                "valueType": "entityState"
            },
            {
                "property": "topic",
                "propertyType": "msg",
                "value": "",
                "valueType": "triggerId"
            },
            {
                "property": "data",
                "propertyType": "msg",
                "value": "",
                "valueType": "entity"
            }
        ],
        "x": 200,
        "y": 160,
        "wires": [
            [
                "531a8ede1ea6f42e",
                "c885b3997b7a27e5",
                "46dd8e28bf5da6f9"
            ]
        ]
    },
    {
        "id": "505255aec0a0bb5e",
        "type": "server",
        "name": "Home Assistant",
        "version": 5,
        "addon": false,
        "rejectUnauthorizedCerts": true,
        "ha_boolean": "y|yes|true|on|home|open",
        "connectionDelay": true,
        "cacheJson": true,
        "heartbeat": false,
        "heartbeatInterval": "30",
        "areaSelector": "friendlyName",
        "deviceSelector": "friendlyName",
        "entitySelector": "friendlyName",
        "statusSeparator": ": ",
        "statusYear": "hidden",
        "statusMonth": "short",
        "statusDay": "numeric",
        "statusHourCycle": "default",
        "statusTimeFormat": "h:m",
        "enableGlobalContextStore": false
    },
    {
        "id": "3a531b742507ca90",
        "type": "ha-entity-config",
        "server": "505255aec0a0bb5e",
        "deviceConfig": "577a1f131a83f6e0",
        "name": "LocalAI Image URL",
        "version": "6",
        "entityType": "sensor",
        "haConfig": [
            {
                "property": "name",
                "value": "LocalAI Image URL"
            },
            {
                "property": "icon",
                "value": ""
            },
            {
                "property": "entity_picture",
                "value": ""
            },
            {
                "property": "entity_category",
                "value": ""
            },
            {
                "property": "device_class",
                "value": ""
            },
            {
                "property": "unit_of_measurement",
                "value": ""
            },
            {
                "property": "state_class",
                "value": ""
            }
        ],
        "resend": false,
        "debugEnabled": false
    },
    {
        "id": "c1f022eec4da33cb",
        "type": "ha-entity-config",
        "server": "505255aec0a0bb5e",
        "deviceConfig": "a2f68e60f5c93945",
        "name": "LocalAI Image Button",
        "version": "6",
        "entityType": "button",
        "haConfig": [
            {
                "property": "name",
                "value": "LocalAI Image Button"
            },
            {
                "property": "icon",
                "value": ""
            },
            {
                "property": "entity_picture",
                "value": ""
            },
            {
                "property": "entity_category",
                "value": ""
            },
            {
                "property": "device_class",
                "value": ""
            }
        ],
        "resend": false,
        "debugEnabled": false
    },
    {
        "id": "577a1f131a83f6e0",
        "type": "ha-device-config",
        "name": "LocalA Weather Image URL",
        "hwVersion": "",
        "manufacturer": "Node-RED",
        "model": "",
        "swVersion": ""
    },
    {
        "id": "a2f68e60f5c93945",
        "type": "ha-device-config",
        "name": "LocalAI Image Button",
        "hwVersion": "",
        "manufacturer": "Node-RED",
        "model": "",
        "swVersion": ""
    }
]
8

FYI, this should work…

rest_command:
  get_image_url:
    url: http://192.168.1.195:8080/v1/images/generations
    method: POST
    headers:
      Content-Type: application/json

    payload: > 
      {
      "prompt": "magnificent scenery, wide landscape, sharp and crisp background, very beautiful landscape, fantasy, birdview, best quality, masterpiece, ultra high res, dark blue light, photo, photorealistic, wide view, kkw-ph1, professional shot of house, {{ state_attr('weather.forecast_home', 'forecast')[0].condition }} day, humidity perception is {{ states('sensor.humidex_perception') }}, thermal perception is {{ states('sensor.thermalperception') }}, extreme low-angle shot, realistic photography, ultra detailedmagnificent scenery, wide landscape, sharp and crisp background, very beautiful landscape, old ruins buildings, fantasy, birdview, best quality, masterpiece, ultra high res, dark blue light, cloudy, photo, photorealistic, wide viewl",
      "size": "512x512",
      "model": "landscape-photoreal"
      }

with

    - service: rest_command.get_image_url
      response_variable: image_url

I’d need to see how the response is formatted for further details.

9

thanks petro!

Here’s an example of the responce from your new approach:

content:
  created: 1716999560
  id: 8815f721-79db-4289-9c41-5aa83465a52e
  data:
    - embedding: null
      index: 0
      url: http://192.168.1.195:8080/generated-images/b643703454608.png
  usage:
    prompt_tokens: 0
    completion_tokens: 0
    total_tokens: 0
status: 200