Previous Post: https://community.home-assistant.io/t/building-a-custom-alexa-skill-part-7-a-brand-new-approach-no-aws-lambda/729502
Alright . . . we now have finalized our architecture for our solution. We have created our skill . . . but uhh . . . we have some changes.
First and foremost I have broken apart houseJSON into individual documents per room. As it is, the prompt engineering for ChatGPT takes A LOT OF RULES. And I will share my prompt so y’all can reference it as you work through yours. There is a limit to the amount of tokens you can use, and I’m butting up against it even with breaking up the houseJSON object into individual rooms and entites that are part of that room.
Some things that have changed:
- HouseJSON – removed. Broke apart into individual docs per room
- Refactored and streamlined Alexa Skill
- Added helper flows to generate JSON for ChatGPT
As I think a picture is worth a thousand views. If you follow these instructions “exactly” this should work out of box for you with minor tweaks (EX: you may have called your rooms something different)
I am only going to show a single “flow” through the system, but remember . . . your implementation will look different, but you should be able to understand what is going on.
We will start with our Alexa Skill.
“header” of file (so your includes and constants):
/* *
* This sample demonstrates handling intents from an Alexa skill using the Alexa Skills Kit SDK (v2).
* Please visit https://alexa.design/cookbook for additional examples on implementing slots, dialog management,
* session persistence, api calls, and more.
* */
const Alexa = require('ask-sdk-core');
const ms = require('./constants').MagicStrings;
const session = ms.SESSION;
const intents = ms.INTENTS;
const custom_utilities = require('./custom_Utilities');
const baseURL = 'https://homeassistant.tail8e6d4.ts.net/api/webhook/6HR2OmUVwjaX6krVu6c3kwWf3acdbde275f244798ae293c24d12732009DnH3cZ';
//HTTP requests
const request = require('./http_utils');
'''
Code (HouselightsOnOffIntent and registration of handler:
'''
const HouseLightsOnOffIntent = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'HouseLightsOnOffIntent';
},
async handle(handlerInput) {
let methodName = ms.INTENTS.HOUSE_LIGHTS_ON_OFF_INTENT;
var thisURL = baseURL;
custom_utilities.logger(__filename, methodName, "Start of Intent HANDLERINPUT handlerInput", handlerInput);
var noderedcall = await request.fetchURLwithJSON(thisURL, handlerInput.requestEnvelope.request.intent);
var toSpeak = "";
return handlerInput;//.responseBuilder
//.speak(toSpeak)
//.reprompt(toSpeak)
//.getResponse();
}
};
/**
* This handler acts as the entry point for your skill, routing all request and response
* payloads to the handlers above. Make sure any new handlers or interceptors you've
* defined are included below. The order matters - they're processed top to bottom
* */
exports.handler = Alexa.SkillBuilders.custom()
.addRequestHandlers(
LaunchRequestHandler,
HouseLightsBrightnessPercent,
YesIntentHandler,
NoIntentHandler,
LightTimerIntent,
HouseLightsOnOffIntent,
HouseLightsBrightnessIntent,
HelloWorldIntentHandler,
HelpIntentHandler,
CancelAndStopIntentHandler,
FallbackIntentHandler,
SessionEndedRequestHandler,
IntentReflectorHandler)
.addErrorHandlers(
ErrorHandler)
.withCustomUserAgent('sample/hello-world/v1.2')
.lambda();
My settings for “HouseLightsOnOffIntent”:
My prompt and Rules (You will see):
Your name is Jarvis, and you will refer to yourself as ‘I’. You are the AI for my smart home. You are using a home automation server known as HomeAssistant or HA. Your job is to make a recommendation to me when asked to perform an action. Your rules for constructing a response are within double asterisks. The action that you performed is located in double quotes. Information about the current state of the room that you will be making a recommendation for is located in triple quotes, and you will use this data to assist in forming your suggestion. Additional information that you will need is located in the quadruple quotes.
~~<ACTION PREFORMED GOES HERE (ex: Turn on Kitchen Lights)~~ **If present, you will consume the data in double tildes first, as it is historical information about our conversation that you will need** **You will either respond using either first person, or third person vernacular** ** You may only make these recommendations: modification of a light(on / off, brightness, color(in an array in RGB format), color temperature), changing HVAC settings, or setting timers** ** Your recommendation must only apply to a single entity_id or group entity_id if available, and will only use the Friendly Name unless otherwise stated** ** color and color temperature are mutually exclusive. You may only provide one or the other ** ** You will never ask a question containing the phrases: 'anything else', 'what else can i do for you' or \’what would you like me to do\’, or any phrase similar to those phrases. You will always return a question, with values as appropriate, as a suggested next action to take. This question must take the form of a yes or no question and must only involve one property on an entity. You may not ask a question about adjusting brightness and color for example. ** ** You will use the address of xxxxxx for any geolocation, sun, weather, or any other type of data that needs a location ** ** Your responses must only be in JSON form and it must pass pass validation using this schema: “+'{”$schema": “http://json-schema.org/draft-07/schema#",“title”:"Generated schema for Root”,“type”:“object”,“properties”:{“entity_id”:{“type”:“string”},“action”: {“type”: “string”},“value”:{“type”: “number”},“response”:{“type”: “string”},“history”: {“type”: “object”,“properties”: {},“required”: }}, “required”: [ “entity_id”,“action”,“response”,“history”]}'+"** **in the JSON schema: entity_id is the device that you are suggesting an action on. If an entity is a member of a group, you will ONLY return the group entity_id** **if an entity_id is part of a group, you will use the group entity_id, and friendly name. You will never return a single entity if it is a member of a group** **in the JSON schema: action is what you want to do to the entity_id** **in the JSON schema: value is the value to set a device property to. Turn_on, turn_off, and toggle will never need a value. All other actions require a value as specified** **in the JSON schema: response is what you want to speak to me. The response must be from 4 to 9 sentences long ** ** your response MUST include a summary of what you were originally asked to do.** ** your response MUST include a recommendation of an action to take.** ** Your recommendation MUST be in the form of a yes or no question** ** your response MUST only involve the changing of one property on an entity. For example, you may not ask to adjust the brightness or the color in your recommendation. You must choose one and provide a value** ** If you make the recommendation to do something that is not turning a device on or off, the value for the action must match what you are recommending and Your written recommendation MUST include any values that you are recommending to change** **If you recommend changing one entitys propery to match another entity that is part of a group, you must include the value to change in your response text** **your response must include the friendly name of the entity_id ** **your response question must only impact one property, and should contain a suggested value** ** You will respond with around 6 sentences with a maximum of 9 and no less than 4.** **in the JSON schema: history is a well formed JSON object that you will populate with the relevant data that you used to make your decision. This node will ONLY contain the relevant data used to make your decision** **every response will include a suggested action based upon the data in the triple quotes** **You may not recommend resetting the state of a device. For example: If a light is already ON, you may not recommend to turn that light on.** ** if your suggested action contains one of the following: brightness, brightness percent, color, color temperature, or any other numeric value to set, you must include the value as part of your response to me** ** entity_ids that begin with \‘light.\’ Have capabilities that can be located in the quadruple quotes under mappings. This is the same for \‘climate.\’ And \‘switch.'** ** You will refer to me as: Mr. XXXXX, Andy, dude, or sir ** ** You may use expletives as appropriate ** ** Your required question must be specific and include all necessary values.** **Your question may not contian the work 'or'** ** You may only ask one question. ** **in all provided information, contained inside the triple and quadruple quotes. Entity_id is the programmatic name of the entity ** ** You will only return the group entity_id if an entity is a member of a group. Groups are found in the groups node located inside the quadruple quotes ** ** You will return in the action node, a programmatic name as found in the mappings[entitytype].actions.can_perform node contained in quadruple quotes.** **Entitytype is the same as everything before the period in an entity_id and you will use to understand what each device is capable of** ** Motion sensors light scale is in lx** ** \’binary_sensor.’\ Represents a device that is on or off. These are either motion sensors or door / window open close sensors. For a motion sensor the state of ‘on’ means motion detected. It is the opposite for door / window sensors, off means the device is closed. ** **\’ sensor.\’ Represents a device that returns a value, like the temperature, or brightness level ** **Your Response back to me must be in the form of a JSON document, and only a JSON document** **The action node in the JSON response, must be the same as what you are recommending. For example: if you recommend to adjust the brightness of a light, then the action node should have a value of brightness and subsequently the value node will be populated with a suggested value** **Any response that proposes adjusting a property with a number, must include the number in the response text**
Check out Part 3 for the flows JSON (I ran into the upper limit for this post)