Saving mqtt sensor values as text file to server

Configuration
,
1

Hi!
I have a IoT application where I measure temperature in the sea. The connection with the gateway works perfectly. MQTT broker is used in HA and sensor.Vattentemperatur_1 shows nice correct temperatur.

mqtt:
  sensor:
    - name: "Vattentemperatur_1"
      state_topic: "Temperatur"
      suggested_display_precision: 1
      unique_id: LSN50_1
      unit_of_measurement: "°C"
      value_template: "{{ value_json.TempC1 }}"

In my case I need to save the temperature value, every time it changes, as a .txt file in a directory on an internet server for further use in an other application.

Made this rest_command in configuration.yaml

rest_command:
  send_temperature:
    url: "https://wwww.server.se/t2"
    method: POST
    payload: "{{ states.sensor.vattentemperatur_1.state }}"
    content_type: "text/plain"
    username: "user"
    password: "passw"

In automation.yaml I have:

- id: '1701867996210'
  alias: Send Temperature to Internet Server
  trigger:
    platform: state
    entity_id: sensor.Vattentemperatur_1 message
  action:
  - service: rest_command.send_temperature
  mode: single

When running the automation I get the respons in HA LOG:
Error. Url: https://wwww.server.se/t2. Status code 500. Payload: b’21.00’

The temperature value 21.00 is correct but there is a b before as I assume means binary. How can it be? The server assumes it should be a “text/plain” value, I guess?

I have struggled a lot with this and I’m on the limit to give up. What ever I do there is this b. Or do I misunderstand the error message? Maybe there are other errors?

Would really appreciate some help with this.

//Lars

2

I have a bash script which takes a MQTT payload and logs it to a text file from cron - similar actions, but not quite the same technology…

#!/bin/bash
# subscribe to a topic waiting for ONE value, print only the payload, use tr to strip " quotes:
LOG_STRING=$(mosquitto_sub -C 1 -q -d -i "monitor"  -h BROKER --username 'UNAME' --pw "SECRET" -v -t 'your/topic/here' -F %p | tr -d '"')

/bin/echo "$LOG_STRING" >> log.txt

This runs from cron on the server which has the logfile, and connects back to the broker remotely.

Sorry - I’ve not used REST, so if the {{ template }} works in Dev Tools, I’m out!

3

Interesting, are running this script in HA?

4

What is the script running on the remote server actually expecting though?
Typically you would be posting stuff to a remote server using multipart/form.

As an alternative what about this:

shell_command:
  curl_upload_data: curl -X POST -H "Content-Type: text/plain" -u "user:password" -d "{{ states('sensor.vattentemperatur_1') }}" https://www.server.se/t2

Then you can just call it in your automation with:

- service: shell_command.curl_upload_data

5

No, the small script is on a separate Linux box and triggered by cron (technically, likely a systemd unit pretending to be cron!).

I’d suggest testing in small parts of the process with mosquitto_pub, mosquitto_sub, and curl. Break things down and test them.

6 
shell_command:
  curl_upload_data: 'curl -X POST -H "Content-Type: text/plain" -u "user:password" -d "{{ states(''sensor.vattentemperatur_1'') }}" https://www.server.se/t2'

If I use single quotes it works in some way…

No error message “Check configuration”
No error message in LOG
No error from “server” to HA

But noting uploaded?? Confused!

Checking Logbook:

log book 1

log book 2
log book 2649×509 26.3 KB

7

What do you mean with a “separat Linux box”?

8

You need to test whether the curl command even works in a terminal first. Like I said previously - I would have expected that you would need to post it as form fields, rather than just uploading a value as text/plain.

But without knowing what the script on the remote server is expecting, it’s difficult to advise further.

9

You are right. I have tested the curl and it doesn’t seem to work.
Unfortunately my knowledge is very limited. I was so happy that i succeeded with the MQTT data and having the sensor working so I dived into too deep water by just thinking, ha! this last part to save the last registered value to a server must be simple… :upside_down_face:

I don’t know what you mean by “post it as form fields” :anguished:
So annoying :unamused:

10

So typically - many servers expect data to be POSTED as if it was coming from a web form.

curl -X POST https://reqbin.com/echo/post/form
   -H "Content-Type: application/x-www-form-urlencoded" 
   -d "param1=value1"

Where param1 would be the name of the field it expects the data to be in, and value1 would be the data.

11

Where param1 would be the name of the field it expects the data to be in, and value1 would be the data.

But I only have a temperature value like 21.5 to save :worried:

12

As I said - without knowing any more about how the server you are posting to works, it’s impossible for me or anyone else to help you with that part.

13

Then it’s difficult. I don’t know how it works. Asked the ISP and the only respons I got was: We don’t give out any information. You are supposed to use sftp if you want to transfer files.
When using sftp I must have a local file to transfer which I don’t have.

14 
shell_command:
  store_sensor: 'echo {{ states(''sensor.vattentemperatur_1'') }} > /config/vattentemperatur.txt'

That will create a text file with just the sensor data in it that you can use to ftp to the server. Now you just have to figure out the ftp part.

15

Tried this:

shell_command:
  store_and_upload_sensor: 'echo "{{ states(''sensor.vattentemperatur_1'') }}" > /config/vattentemperatur.txt && curl --user "user:passw" -T "/config/vattentemperatur.txt" sftp://path/to/t2'

No config errors or LOG errors in HA but nothing uploaded.

Tested curl command from command line in a folder with a file “temperature.txt”

$ curl --user "user:passw" -T "temperatur.txt" sftp://path/to/t2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:01:20 --:--:--     0^C

No errors but had to do ^C, nothing happened.
:cry:

If I do the same operation in FileZilla instead of using curl it works smoothly.

16

Yeah… never seen curl used that way before. You would need a command line ftp program to upload it, NOT curl.

A quick google around suggests you need to do something like this:

sshpass -p [password] sftp user@host:/path/to/remote/file <<EOF
put /local/path/to/temperature.txt
exit
EOF

or

sshpass -p [password] scp -P 23 /local/path/to/temperature.txt user@host:/path/to/remote/file/
17

Thanks for your patiens :blush:

SCP+ssh! Just for the moment I actually use scp in a “manual” command line update for the temp and time files.

scp temperatur.txt tid.txt [email protected]:/path/to/t2
always works perfectly but I have to manually add password.
But I don’t think SCP is supported in HA :anguished:

18

Pretty sure scp is part of linux, so it probably is in the image for Home Assistant, as for the password - yup, that’s why you are supposed to use sshpass command before the command that will require a password. The proper way to do it though is to generate an ssh key on the server and copy it to your home assistant box so that homeassistant can log straight in to the server with the key, and skip needing a password.

19

Nice! :grinning:

Used notify to save the temperature data locally. Works well.
The automation triggers this “notify” to run each time sensor has changed.

notify:
  - platform: file
    name: Vattentemperatur
    filename: /config/data/Vattentemperatur.txt
    timestamp: False

- id: '1701867996210'
  alias: Save Temperature to Local File
  description: ''
  trigger:
  - platform: state
    entity_id: sensor.vattentemperatur_1
  action:
  - service: notify.Vattentemperatur
    data_template:
      message: ' {{ states.sensor.vattentemperatur_1.state }}'
  mode: single

Installed sshpass i Linux and copied the file to the server with good result.

Also tried scp with ssh in HA and it was supported. Wrote password manually and the file “Vattentemperatur.txt” was uploaded :+1:

Don’t know how or if sshpass is possible to install and use in HA (docker/container??). I can live with the risk of exposing the password as done in sshpass.

Read the file about setting up ssh keys. Many steps with possibilities to do mistakes. Have to think about that :grimacing:

1 Like
20

I have:

shell_command:
  upload_script: bash /config/data/upload.sh

and

- id: '1702997888249'
  alias: Upload script
  description: Upload to server
  trigger:
  - platform: time_pattern
    minutes: /10
  condition: []
  action:
  - service: shell_command.upload_script
    data: {}
  mode: single

upload.sh (executable, tested, works)

#/bin/bash
sshpass -p [passw] scp -P 22 Vattentemperatur.txt [email protected]:/path/to/t2

No errors concerning script, automation and so on but a new error:

Error running command: bash /config/data/upload.sh, return code: 127

I guess it has something to do with:

If you are using Home Assistant Operating System, the commands are executed in the homeassistant container context. So if you test or debug your script, it might make sense to do this in the context of this container to get the same runtime environment.

So close (I think) but have no idea to come around this. Read several articles about this return code 127 but don’t understand how to solve this “container issue” :worried: