Convert hex string to base64

Configuration
1

I have a hex string like “010203”

I need to treat that string as bunch of hex bytes (0x01, 0x02, 0x03), not an ACSII string.

If I use "{{ '010203' | base64_encode }}"

I get MDEwMjAz which decodes to 303130323033 which is the ASCII

How to I do a ‘binary’ base64_encode?

2

Maybe I’m just not following.
But binary is 0 and 1.
base64 means there are 64 characters in the to represent the numbers.

So what is binary base64?

3

Yeah - I agree. Maybe binary was not the best word.

Maybe ‘bytes’ → Base64 is a better term

4

it decodes to “010203” if you use base64_decode. I think you need to come back with the correct terminology or provide the expected answer so we can reverse engineer what you really want.

EDIT: Are you saying you want 010203 to change to 0x01 0x02 0x03?

5

Let me clarify again

I have a string “010203”, that represents a 3-byte array = [1,2,3]

I want to convert that to this string “AQID”

For an online equivalent:

Enter 010203 in the HEX input

6

is each pair base 16?

7

yes

for another example:

Input: “0F010003” = [0x0F, 0x01, 0x00,0x03]
Output “DwEAAw==”

8 
{{ (int('0F010003', base=16) | pack(">I")).decode('utf-8') | base64_encode }}

EDIT: I Had to “jump through hoops” to get this to work because the jinja functions don’t work with bytes. Kinda annoying. Might put in a PR to address that.

EDIT2: Added 2 PRs to make this easier, will be a couple months before they are merged though.

which will result in a “simple” template: {{ '0F010003' | fromhex | base64_encode }}

1 Like
9

Oh!

Thanks. Will give it a try tomorrow. Getting late here.

10

How do I use your answer to encode a hex string that is an input field to a script

Assume {{ command }} is a string like '0F010003'

sequence:
  - variables:
      command64: >-
        "{{ (int( "{{ command }}", base=16) | pack('>I')).decode('utf-8') |
        base64_encode }}"

Thats is not working - I think because I have nested {{ {{ }} }}

11 
sequence:
  - variables:
      command64: >-
        {{ (int(command, base=16) | pack('>I')).decode('utf-8') | base64_encode }}
12

I am having problems when running this script.

Here is the full script.

The error is: failed to perform the action script/settempalarm. UndefinedError: 'None' has no attribute 'decode'

sequence:
  - variables:
      command64: >-
        {{ (int(command, base=16) | pack('>I')).decode('utf-8') | 
        base64_encode  }}
  - action: mqtt.publish
    metadata: {}
    data:
      qos: "0"
      retain: false
      topic: application/{{ applicationID }}/device/{{ device }}/command/down
      payload: >-
        { "devEui":"{{ device }}","confirmed": false,"fPort": 1,"data":"{{
        command64 }}" }
fields:
  device:
    selector:
      text: null
    name: device
    description: DevEUI
    required: true
  applicationID:
    selector:
      text: null
    name: applicationID
    description: Chirpstack Application ID
    required: true
  command:
    selector:
      text: {}
    name: Command
    description: Hex String
    required: true
alias: SendCommand
description: Sends MQTT message to the Gateway with a downlink command
13

Are you putting in a command when you test it? That error tells me you aren’t.

14

Yes: command is set (and is “AA01000200150020”)

If I edit the code as below, all runs fine and I get an MQTT Publish

sequence:
  - variables:
      command64: "{{ command }}"
  - action: mqtt.publish
    metadata: {}
    data:
      qos: "0"
      retain: false
      topic: application/{{ applicationID }}/device/{{ device }}/command/down
      payload: >-
        { "devEui":"{{ device }}","confirmed": false,"fPort": 1,"data":"{{
        command64 }}" }
fields:
  device:
    selector:
      text: null
    name: device
    description: DevEUI
    required: true
  applicationID:
    selector:
      text: null
    name: applicationID
    description: Chirpstack Application ID
    required: true
  command:
    selector:
      text: {}
    name: Command
    description: Hex String
    required: true
alias: SendCommand
description: Sends MQTT message to the Gateway with a downlink command

MQTT:

{
  "devEui": "a840414e4f5cae3d",
  "confirmed": false,
  "fPort": 1,
  "data": "AA01000200150020"
}
15

Try this then, FYI you swapped the struct from 4 bytes to 8, which is why it was failing. I.e. Your original question used 3 bytes (010203), your example had 4 ( 0F010003). Now you’re trying to pass 8 (AA01000200150020)in your last post.

sequence:
  - variables:
      command64: >
        {{ (int(command, base=16) | pack('>Q')).decode('utf-16') | base64_encode }}
  - action: mqtt.publish
    metadata: {}
    data:
      qos: "0"
      retain: false
      topic: application/{{ applicationID }}/device/{{ device }}/command/down
      payload: >-
        { "devEui":"{{ device }}","confirmed": false,"fPort": 1,"data":"{{
        command64 }}" }
fields:
  device:
    selector:
      text: null
    name: device
    description: DevEUI
    required: true
  applicationID:
    selector:
      text: null
    name: applicationID
    description: Chirpstack Application ID
    required: true
  command:
    selector:
      text: {}
    name: Command
    description: Hex String
    required: true
alias: SendCommand
description: Sends MQTT message to the Gateway with a downlink command

EDIT: If that isn’t right, then changes need to happen.

16

I will not know in advance how long the input hex string is.

I will try with your change … but what if I have an even longer string (say 20 hex bytes?)

17

Do you have a link to the documentation for whatever device we are dealing with here?

18

HA is not currently equipped with the ability to do this. You’ll have to wait until my 2 PRs above are in the software.

19

Thanks for all your help so far.

Regarding the longer input strings: When encoding HEX → base64, I might try chunking the input into 3-byte chunks (which will always give 4 base64 chars) and then join the chunks together.

Question: I have the option of NOT using a string that represents the input bytes; I could use a byte array instead. I wonder if that makes the code easier to handle. I will do some checks.

You asked what equipment I am dealing with and a link to the docs.

The project is LoRaWAN. I have this common LoRaWAN gateway.

This device uses MQTT to:

  • report remote sensor data (that part is easy)
  • receive commands to be forwarded to the remote sensor

The sending commands documentation is here

This is the important part of that MQTT message:
"data": "...." // base64 encoded data (plaintext, will be encrypted by ChirpStack)

And the ‘data’ is a bunch of bytes that is a command. For example the command to

  • turn on alarm mode
  • Check temp every 2 mins
  • set the alarm temp range as being outside 16C and 32 C is AA01000200100020

Another command is Reset = 04FF

I will experiment with these possible solutions:

  • Converting byte-array instead of string
  • 3-byte chunking
20

Just realised a problem with your suggested code:

{{ (int('AA010003', base=16) | pack(">I")).decode('utf-8') | base64_encode }}

Results in
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaa in position 0: invalid start byte

Indeed 0xAA is not a valid utf-8 character.

Is there a different .decode(xx) option?

I seached but had trouble finding the documentation on the decode filter