Voice + Perplexity

Configuration Voice Assistant
,
1

I would like to expand a little on the topic of HA Voice. Overall, HA Voice works well and I have no complaints. However, as a Nabu Casa subscriber, I would expect a little more from the assistant than just turning the lights on and off. I have been struggling with this for a long time and still can’t get it to work. My connection HA with Perplexity dosent work.
The assistant is still called Nabu, but I have named the information network, i.e., the internet, “the box.” I have a script for this

> alias: Perplexity - Ask and Read (CORRECTED)
> description: Script for asking questions to Perplexity AI and reading the answers
> mode: single
> icon: mdi:robot-happy
> fields:
> query_text:
> description: Question for Perplexity AI
> example: What is the weather like in Szczecin?
> required: true
> selector:
> text: null
> sequence:
> - alias: Set question in input_text
> target:
> entity_id: input_text.perplexity_question
> data:
> value: "{{ query_text }}"
> action: input_text.set_value
> - alias: Send query to Perplexity API
> action: rest_command.perplexity_query
> data:
> query: "{{ query_text }}"
> response_variable: api_response
> - alias: Save the response
> target:
> entity_id: input_text.perplexity_answer
> data:
> value: "{{ api_response.content.choices[0].message.content }}"
> action: input_text.set_value
> - alias: Read the response via TTS
> data:
> entity_id: media_player.home_assistant_voice_09641e_media_player
> message: "{{ states('input_text.perplexity_answer') }}"
> action: tts.google_translate_say
> - alias: Create notification with answer
> data:
> title: Perplexity AI
> message: |
> Question: {{ query_text }}
> 
> 
> Answer: {{ states('input_text.perplexity_answer') }}
> action: persistent_notification.create

and an endless list of queries my custome sentence

> ```
> ```
> language: "pl"
> intents:
>   AskPerplexity:
>     data:
>       - sentences:
>           - "zajrzyj do pudełka {query}"
>           - "sprawdź w pudełku {query}"
>           - "pudełko {query}"
>           - "zajrzyj do pudełka i {query}"
>           - "sprawdź w pudełku co wiesz o {query}"
>           - "pudełko powiedz mi o {query}"
>           - "zajrzyj do pudełka i sprawdź {query}"
>           - "pudełko co wiesz o {query}"
>           - "zapytaj pudełko {query}"
>           - "sprawdź w pudełku informacje o {query}"
>           - "zajrzyj do pudełka i wyjaśnij {query}"
>           - "sprawdź w pudełku dane o {query}"
>           - "pudełko wyjaśnij {query}"
>           - "pudełko daj informacje o {query}"
>           - "zajrzyj do pudełka i daj mi szczegóły o {query}"
>           - "sprawdź w pudełku szczegóły {query}"
>           - "pudełko opowiedz o {query}"
>           - "zajrzyj do pudełka i opowiedz o {query}"
>           - "sprawdź w pudełku fakty o {query}"
>           - "pudełko jakie są informacje o {query}"
>           - "zajrzyj do pudełka co możesz powiedzieć o {query}"
>           - "sprawdź w pudełku co wiesz na temat {query}"
>           - "pudełko jakie masz dane o {query}"
>           - "zajrzyj do pudełka i sprawdź dane o {query}"
>           - "sprawdź w pudełku co jest ważne o {query}"
>           - "pudełko co warto wiedzieć o {query}"
>           - "zajrzyj do pudełka i podaj informacje o {query}"
>           - "sprawdź w pudełku jakie są fakty o {query}"
>           - "pudełko przedstaw {query}"
>           - "zajrzyj do pudełka i przedstaw {query}"
>           - "sprawdź w pudełku co mówią eksperci o {query}"
>           - "pudełko jakie są zalety {query}"
>           - "zajrzyj do pudełka i powiedz zalety {query}"
>           - "sprawdź w pudełku wady {query}"
>           - "pudełko czym jest {query}"
>           - "zajrzyj do pudełka i wyjaśnij czym jest {query}"
>           - "sprawdź w pudełku jak działa {query}"
>           - "pudełko jak działa {query}"
>           - "zajrzyj do pudełka i sprawdź jak działa {query}"
>           - "sprawdź w pudełku definicję {query}"
>           - "pudełko jaka jest definicja {query}"
>           - "zajrzyj do pudełka i podaj definicję {query}"
>           - "sprawdź w pudełku znaczenie {query}"
>           - "pudełko co oznacza {query}"
>           - "pudełko kto {query}"
>           - "zajrzyj do pudełka i sprawdź kto {query}"
>     slots:
>       query:
> ```

, about 255 ofcors i give u sample in PL language. When I ask, “Hey Nabu, look in the box and check what the weather is like in Boston,” nothing happens. I only get a response saying that there is no access and it can be checked in internet, and so on.
Ofcors if u got time u can check my adds to config yaml

# ======================================================================
# POLE TEKSTOWE DO PRZECHOWYWANIA PYTANIA DLA PERPLEXITY
# ======================================================================
input_text:
  perplexity_question:
    name: Pytanie do Perplexity
    initial: "Jaka jest pogoda?"
    max: 255
    mode: text

# ======================================================================
# INTEGRACJA PERPLEXITY AI - REST COMMAND
# ======================================================================
rest_command:
  perplexity_query:
    url: "https://api.perplexity.ai/chat/completions"
    method: POST
    headers:
      Authorization: "Bearer pplx-Here is my API" 
      Content-Type: "application/json"
    payload: >
      {
        "model": "sonar",
        "messages": [
          {"role": "system", "content": "Odpowiadaj zwięźle i jasno po polsku w maksymalnie 3-4 zdaniach."},
          {"role": "user", "content": "{{ query }}"}
        ],
        "max_tokens": 200,
        "temperature": 0.2
      }

# ======================================================================
# SENSOR PERPLEXITY AI - ODBIERANIE ODPOWIEDZI
# ======================================================================
rest:
  - resource: "https://api.perplexity.ai/chat/completions"
    method: POST
    headers:
      Authorization: "Bearer pplx-ZDRNLz API L7EvPqJFV"
      Content-Type: "application/json"
    payload: '{"model":"sonar","messages":[{"role":"system","content":"Odpowiadaj zwięźle i jasno po polsku w maksymalnie 3-4 zdaniach."},{"role":"user","content":"{{ states(''input_text.perplexity_question'') }}"}],"max_tokens":200}'
    scan_interval: 30
    sensor:
      - name: "Perplexity Answer"
        unique_id: perplexity_answer_sensor
        value_template: >
          {% if value_json.choices %}
            {% set content = value_json.choices[0].message.content %}
            {% if content|length > 250 %}
              {{ content[:250] }}...
            {% else %}
              {{ content }}
            {% endif %}
          {% else %}
            Brak odpowiedzi
          {% endif %}
        json_attributes_path: "$.choices[0].message"
        json_attributes:
          - content
          
# ======================================================================
# CONVERSATION - ASYSTENT GŁOSOWY
# ======================================================================
conversation:
2

Have you tried using the OpenRouter integration rather than the Perplexity API?

3

Nic, Nice never heard about this i try. Thanks

4

I haven’t used it myself because I already have a paid-for Perplexity account (not integrated with HA) and you can’t use the native Perplexity API with OpenRouter - be interesting to hear how you get on.

5

I made integration with free Nvidia AI wit Openrouter web site works perfect but I did integration w Openrouter and HA. Now when i try to ask something for example weahter i get answear:

I don’t have access to current weather data because my tools don’t include a direct connection to meteorological services. I recommend checking a weather app or website that provides forecasts for Szczecin. Do you need help with anything else?

or:

What do you know about The Witcher movie?
Details about The Witcher movie cannot be retrieved at this time due to a technical error. I suggest checking its availability on streaming platforms or databases such as IMDb. Can I help you with anything else?

I also have a question: in my case, can these two assistants, HA and Openrouter, be called up with a single command, e.g., “Hey Nabu,” and automatically select the stream? Right now, I have them split between Nabu and Jarvis. I understand how it works, but other users don’t want to complicate things even more.`

6

I’ve only been able to use two services in custom sentences - then you can specify the LLM you want to use in the intent,

CustomGoogle:
  action:
    - choose:
        - conditions: "{{ is_state('binary_sensor.online', 'off') }}"
          sequence:
            - action: script.tts_response
              data:
                tts_sentence: "Sorry. There's no internet at the moment."
        - conditions: "{{ is_state('binary_sensor.online', 'on') }}"
          sequence:
            - action: script.tts_response
              data:
                tts_sentence: "I'll check with Google."
            - action: conversation.process
              data:
                agent_id: conversation.google_ai_conversation
                text: "{{ question }}"
              response_variable: api_response
            - variables:
                answer: "{{ api_response.response.speech.plain.speech }}"
            - action: script.tts_response
              data:
                tts_sentence: "{{ answer }}"

If you want an AI that searches web pages, the Google Gemini integration will do it - it would answer both the questions in your example.

7

Nop it dosent work 4 me