Nextion Handler for Home Assistant (for NSPanels)

krizkontrolz · February 20, 2022, 2:04pm

Like quite a few other people, I’ve been working on integrating my Sonoff NSPanels into Home Assistant. But the approach I’m using is a bit different from what I’ve seen others doing - it uses a Python script to provide a service so that you can send instructions to Home Assistant directly in the Nextion Editor.

This allows you to program a Nextion touch screen device (NSPanels in particular) to interact with Home Assistant (HA) without having to do any coding in ESPHome YAML or Home Assistant automations.

The diagram below summarises how it works, and I’ve listed the instruction set that nextion_handler makes available in the Nextion Editor below. There are only 3 places in your HMI files that you need to add customized code (highlighted in light blue in the diagram) - the rest can use standardized template code.

I have shared more details on GitHub together with some example files that can be used as templates to get going.

Nextion handler framework

(Updated: version 0.7beta - 2022-07-24)

SET COMMANDS (fetch HA data to update the Nextion)

(Nx = Nextion variable name; E is Home Assistant entity_id).

sett Nx len E (assign len chars of state of E, as string/text, to Nx).
setn Nx scale E (d) (assign Nx the integer value of scale * state of E)
setb Nx E (assign Nx 0 or 1 based whether the state of E is in FALSE_STATES)).
setb Nx E cp x (assign Nx 0 or 1 based on comparing (cp) the state of E to x).
setlt Nx_state Nx_tp Nx_brt Nx_ct Nx_rgb565 E (assign Nx variables the state, ‘type’ (supported modes), brightness, color temperature and color of light E ).
setmp Nx_vars E (assign state and attributes of media_player E to the list of Nx_vars)
setcl Nx_vars E (assign state and attributes of climate E to the list of Nx_vars)
setntf Nx_count (Nx_title) (Nx_msg) (n) (chars_title) (chars_msg) (assign 3 Nx variables the Count, Title and Message of the nth Persistent Notification).
setdt Nx (assign Nx current data-time as “dd/mm HHhMM”).
setsys Nx_vars (assign system data to the list of Nx_vars).

ACTION COMMAND LIST (called by Nextion events to perform actions in HA)

tgl E (toggle E).
ton E (turn on E).
tof E (turn off E).
inps E string (set value of input_select E to string).
inpb E b (turn input_binary E on if b!=0 otherwise turn off).
inpn E x (set value of input_number E to x).
scn E (turn on scene E).
scpt E (call script E).
say E string (Play TTS of message string to media player E).
ntf string (Display a persistent notification with message string to HA).
ntfx n (Dismiss the nth Persistent Notification in HA).
sub Nx (click Nx,1 the Nextion (hidden) hotspot Nx to execute a ‘subroutine’).
cl svc E args (call the svc service for climate E with the listed args).
cv svc E args (call the svc service for cover E with the listed args).
lt svc E args (call the svc service for light E with the listed args).
mp svc E args (call the svc service for media_player E with the listed args).

(For full details of what each Set and Act command does, see the doc strings for functions with the same names in the Python script, including the adjust() helper function that helps make delta/absolute/percentage changes to attributes.)

krizkontrolz · April 2, 2022, 1:08pm

Nextion Handler version 0.5 is now available.

The 3 new main features this version adds are:

New commands for controlling your Lights (setlt, lt_brt, lt_ct, lt_rgb, lt_hs, lt_cw, lt_wt), including color wheel control and a generalized ‘pop-up’ page of light controls (use as a template).
New commands for working with Persistent Notifications (setntf, ntf, ntfx), including a ‘pop-up’ page for notifications (use as a template).
A robust implementation of ‘swipe’ and ‘press’ touch GESTURE controls on the Nextion (which will likely to be useful to anyone still trying to get these to work reliably in their own projects).

I have also completed the documentation on GitHub, so it should be much easier to understand the example HMI files and to create/adapt your own.

The approach is working very well on my own devices (example below):
(updated to Minimalist-style design 29 May 2022.)

krizkontrolz · April 23, 2022, 5:34am

There are some parts of this project that might be of wider interest to other Nextion users in designing their own interfaces to Home Assistant. So I’ve made a folder in my GitHub project to share some of these tips, tricks, and traps with a demo Nextion HMI file and some documents explaining them. This includes:

Gestures: simple and more advanced ways of getting swipe and press gestures to work reliably (especially dealing with some of the unexpected touch behaviours and interactions);
Dynamic Round/circular sliders: a demonstration of some of the more advanced things that can be done with realtime processing of touch events (while duplicating the circular slider that comes on the original NSPanel firmware - see below);
Geometric functions: arctan and square root approximation subroutines needed for interpreting touch events for some custom UI components.

ROUND-SLIDER_Progressive-Anim

The demo/debug pages provide real-time data updates during touch interactions so that you see how touch information is being generated and used (in both expected and unexpected ways). In most cases you can explore what is going on just by opening the page in the Nextion Editor and running it in the simulator. And the markdown documents include some of the key blocks of code (if you just want to check that out without downloading the HMI file).

krizkontrolz · May 29, 2022, 12:05pm

( Updated 8 June 2022: v0.6 now uploaded to repository, including installation instructions and feature summary for the new Widget UI option. )

Over the past 2 months I’ve added information to the project Github page on designing UIs for NSPanels (and other HMIs), both for the touch interactions and the graphical UI to go with them.

The UI_Design section includes tips for others designing their own custom UIs and includes very detailed design rules for an adaption of the Minimalist UI style that has been become popular on these forums (tben’s Lovelace UI • Minimalist and piitaya’s Mushroom Cards ). With a little patience anyone can create beautiful UIs for their HMI projects by methodically following and adapting the Minimalist Design guidelines. A template vector graphics (SVG) file, including UI components that can be adapted and snapped together) and example Nextion Editor (HMI) file are included in the repository. (The screenshots above have been updated to show some example pages using these design rules.)

For those who would prefer the convenience of a simpler option to integrate their NSPanels into Home Assistant, I have added a simplified ‘Widget UI’ option that is extremely easy to use, without any complex custom programming or configuring. The Widget UI blends elements of the original NSPanel widgets, the Home Assistant ‘glance’ cards, and some of the graphical and functional aspects of Mushroom cards and Minimalist UI. After installation, customising your Widgets is as simple as setting up an HA glance card - you simply provide a list of entities (and optionally a name and icon to go with them - but you can ultimately customise all the information displayed on the cards, including template Cards where you can use HA templates to dynamically control everything). Pages are tiled with a set of adaptable generic cards that are populated sequentially with information from your list of HA entities. As shown in the demo screenshots below, the information displayed (and touch interactions) adapt to the type of entity being displayed (see Github page for details of current features and how it works).

krizkontrolz · July 6, 2022, 11:43am

While the configuration of the Widget UI is just a few basic steps (see link above), there are a few people who said they had some initial difficulty with the part about how you set up a URL to a local file on your Home Assistant Raspberry Pi (or other device). Standard Home Assistant installations support this by allowing you, and home assistant services, to access files you put in the /config/www folder without having to enter a username and password.

I suggest keeping the path simple, something like:
/config/www/nspanel/nsp1.tft
which you then access with the URL:
https://MY_HA_URL:8123/local/nspanel/nsp1.tft
This is the URL you enter for tft_url: at the top of the ESPHome YAML configuration.

The https://MY_HA_URL:8123 part of the URL is what you enter into your browser to access your HA interface (e.g. https://mydomain.duckdns.org:8123).

When you download the TFT originally from GitHub, and each time you update it after that, RENAME THE FILE TO MATCH THE TFT URL and then copy/replace it in the proper folder. (This is much easier than changing the TFT URL in the ESPHome configuration each time).

Some checks and tips if you are having trouble with setting this up:

Check that the https://MY_HA_URL:8123 part of the URL opens the Home Assistant UI (fix this part first);
Then enter the full TFT URL https://MY_HA_URL:8123/local/nspanel/nsp1.tft into the same browser - it should download the TFT file (nsp1.tft in this case). If you can’t access the file at this URL, ESPHome (or any other HA service) won’t be able to access it either;
Check the local filename and folder match the URL, /config/www/nspanel/nsp1.tft in this case. Note that you do not create a 'local' folder in the file path (HA makes files in the /config/www folder available at https://MY_HA_URL:8123/local);
Check that the basic homeassistant: section of your HA configuration.yaml file has not changed default access locations/restrictions for files, such as in the allowlists.
If you have not set up secure HTTPS access to your Home Assistant (!!), then you will likely have to use local HTTP access instead at http://homeassistant.local:8123/local or http://my_local_HA_IP:8123/local to access the /config/www folder.

ClaireRovic · July 14, 2022, 9:21am

First I have to say - Thanks Heaps - how quickly has this project moved forward is amazing really. I have four toys to setup throughout the house, so looking forward to the adventure. Before I start I read in these forums that I should let the nspanel do a full firmware update within the Sonoff application before I burn ESPHome - does anyone know if that is still a requirement.

krizkontrolz · July 14, 2022, 4:03pm

Thanks Claire. No, I don’t think that is necessary anymore - I flashed ESPHome to all my NSPanels before ever even booting/powering them up. Just be aware that:

after you physically flash ESPHome the first time to back up that basic configuration file (you will need the password details etc. to configure the template configuration, and if you lose those details you won’t be able to do OTA updates, which would mean needing to reflash physically again);
keep the filename of the ESPHome YAML the same when you copy in the template configuration (the first time and on subsequent updates);

Do you have the US or EU version of the NSPanel? The next version is still being beta tested at the moment and is a big improvement but I’ve been working on the EU version first. PM me if you would like to test the newer beta version (the older 0.6 version works as a proof of concept, but is a bit clunky and has less features).

Blackymas · July 24, 2022, 8:29pm

wow good work. Currently I am still very confused how and where I have to do what. But think your way is the best. With most templates you run into the problem with the flash memory that seems to be different here.

Is there already a HMI for EU NSPanel?

krizkontrolz · July 25, 2022, 8:43am

Thanks @Blackymas, there are both EU and US versions of a much-improved Widget UI currently in beta. It has a much better ‘gesture’ system (tap or swipe and hold on Widget cards for quick controls, with graphical feedback on what action will be performed for that entity) and I’ve just finished adding support (customised dashboard information and interactive gesture controls) for all 36 standard entity types (domains). But I don’t have all of these device types in my HA system, so am looking for more beta testers especially for the new climate.* (with detailed ‘popup’ page), fan.*, humidifier.*, lock.*, and water_heater.* features.

I don’t recommend installing the v0.6 at this stage (it is quite clunky in comparison) - either wait for the 0.7 version or join the beta. If you or anyone else wants to help with final testing, I’ll PM them the details and am happy to help them get it installed and running.

Blackymas · July 25, 2022, 12:00pm

why not - I can try it.

The design is not my cup of tea but the idea and the programming is nice.

Do you have discord? think in discord we can write easier than here

krizkontrolz · July 25, 2022, 2:01pm

PM sent with details for beta.

The HMI files are provided so people can reskin to their liking by replacing the graphics resources with their own designs (and then shifting Nextion UI component locations to match if necessary) in the Nextion Editor. If you want to change fonts, make sure that you create those (with the Font Generator tool) as utf-8 fonts and customise your subset of glyphs to include all those used in the project.

krizkontrolz · July 28, 2022, 4:47am

I’ve set up a Discord group for Nextion Handler that anyone interested in this project can use to ask questions and share ideas.

krizkontrolz · July 30, 2022, 12:40pm

Version 0.7 of Widget UI is now out of beta for general release. Thanks to @zigomatic in particular for feedback and suggestions during testing. This version adds a lot of improvements over the previous version, the highlights of which are:

All 36 standard HA entity types are now supported, with Widget cards automatically adapting their gesture interactions and information displayed to the type (domain, class and available attributes) of the entity.
It uses a new gesture system to interact with the Widget cards with swipes and presses that are modified by how long you hold your finger down (with a gesture indicator that dynamically updates to show the action that will be triggered for that Widget and gesture).
Additional Popup Control pages for more complex types of devices (lights, media players, climate/HVAC devices, notifications and settings).
Works with both EU and US NSPanels.

The installation instructions and other documentation have been updated for the new files and features if you want to check it out or give it a try. The detail in the documentation is ‘folded away’ to make the page easier to navigate - expand the sections with arrows () once you have found the section you need.

Demo of gesture indicator:
NSP_Gesture_Indicator

Betty-G · August 12, 2022, 10:57am

Hello, first of all:
Thank you very much for your work and above all
publish

This can probably also solve the problem with ESPhome/EasyESP, because it’s not just about Sonoff NSPanel, but about Nextion.
(unfortunately many did not understand that NSPanel is also about the display, i.e. flash Nextion)

The HMI is really easy to understand, the designs are beautiful. (I especially like “Neon” very much)

Question:
if the Nextion is to be integrated as a display in HA, where the ESP (32 or 8266) and connected sensors (thermometer) and relays (as switches) have been integrated into HA via MQTT, how should the Nextion display be integrated? Also via MQTT?

I have dozens of Nextion displays (also an intelligent version), I left my NSPanel in the original for the Sonoff devices, I use the “Ewelink integration” in HA.

krizkontrolz · August 13, 2022, 3:11am

No, it uses ESPHome, which has its own more efficient (and secure) protocol for communicating with Home Assistant. And, yes, as you say in your other point there are TWO components on the NSPannel that need flashing - both the ESP32 chip (flashed with ESPHome using the template YAML config.) and the Nextion display (using the HMI/TFT). Then a Python script on Home Assistant runs the command_strings that are programmed into the HMI (either using the pre-compiled Widget UI TFT and/or your own custom HMI pages).

The ESPHome template config. takes care of everything needed to transfer commands from the Nextion to Home Assistant and back the other way. It also takes care of all the local communication between the Nextion display and ESP32 (with all is other peripherals - buttons, relays, temperature sensor, buzzer, rtttl preset tunes, and WiFi) so that the HMI/Nextion can have full local control when there is no WiFi or connection to HA.

Betty-G · August 13, 2022, 3:22am

Hi, thanks for your answer.
I thought to myself that I could transform the projects I have made so far, I fell in love with your illustrations and you wrote “Nextion-Display alle Kind”…

So is it possible to install an ESP32 in ESPHome for your project, plus Nextion editor?

Unfortunately, ESPHome is a bit bitchy when it comes to Nextion, maybe it’s easier for NSPanel…
And if an ESP32 is used for NSPanel, it could actually be installed on any ESP32, right?
Sorry, but before I demolish my Sonoff panel, I want to be on the safe side, so create a “test copy” first…

krizkontrolz · August 13, 2022, 4:38am

I’ve spent a lot of time debugging those issues - the template ESPHome config. and Python script take care of that now, so you shouldn’t have to worry about it. (The main issue is that ESPHome appends \x00 to strings, forwarded to HA, when you write to it with any of the Nextion Custom Protocols - these have to be stripped for subsequent code to work reliably, but that is taken care of).

(I do still have an issue with the commands from ESPHome occasionally/intermittently not triggering the Python script, but I’m working on that - the HMI and ESPHome are working correctly, so I’ve traced it to an issue with how HA is responding to that data after it is received.)

You should be able to adapt your custom HMI code pages by adding the ‘boilerplate’ and ‘custom’ parts of the code (see diagram in first post - you can copy these from a Widget UI HMI) that take care of sending command_strings:

the custom command_strings you program into your HMI event code to perform actions in HA: HA_ACT;
the boilerplate subroutine you use to send HA_ACT commands, after assigning them (and TRIGGER the nextion-handler.py script to process them): SEND_ACTIONS;
the custom command_strings you set to specify what data you want HMI page updates to fetch from HA: HA_SET1, and HA_SET2;
the boilerplate page post-init code to send the page’s HA_SET command strings to HA when you change to that page;
the boilerplate timer that automatically TRIGGERs page updates by fetching the HA data specified in the page’s HA_SET strings: UPDATE_LOOP;
then you will (usually) need a custom APPLY_VARS subroutine to do something with the returned data after the HA_SET commands have finished fetching the updated data - the Python script will automatically call this subroutine whenever it finishes a data update;
some of the other pieces of boilerplate HMI code on page preinit etc. that tie everything together (and any other custom features/code you want to bring across into your own projects) .

Creating your own custom HMI pages is not for the faint of heart - it is not difficult, you just have to work patiently and methodologically. But the rewards are really worth it when you have a completely bespoke UI to your smart home with the exact style and functionality that you want. If you are already comfortable in the HMI editor, then you are a long way there (and that is the only place you have to do any programming, the ESPHome template and Python script take care of the rest).

Betty-G · August 13, 2022, 10:55am

hello, thanks for the replies.
I didn’t quite get the “boiler plate” thing, but I’ll google it and read it…
I know that a few years ago I had a lot of difficulties getting my Nextion devices into HA and did a lot of things within Editor. Luckily there is a “port extension” for the Intelliegent series, with which relays can also be switched directly with Nextion, which can also be defined in Editor, so I brought the connected sensors to HA via MQTT because that was the easiest way for me . The rest, such as GPIO switching was done by Nextion, as well as display dimming, switching on/off as well (Nextion can also do this without an intelligent version)
However, since I saw that there is a way for NSPanel to be demulated, with only one ESP32 being required, I assumed that this way, assuming that the same GPIOs were used, the panels would be “recreated could”.

I have different purposes for Nextion, not as a central one, but each room has its own device on the wall, being “always on” with clock and temperature display. Just like the original NSPanel. When I set up my first displays, I didn’t know that one day there would be exactly the same devices for the same purposes… I also have soil moisture sensors, and not only a heating circuit (thermostat head 230V) but also a small water pump (5V) on the windowsill. So I “designed” almost the same device a few years ago… 2 relays, but with some Nextion without “physical switches”…
Since I had problems in HA, I used EasyESP…

Setting the correct time isn’t difficult, but since I didn’t use a basic version but Enchange or intelligent, and these versions have an integrated clock, I added an additional page in the editor where the time and date can be set , and so, instead of the “Internet clock” the Nextion uses its own “NTC”, which does not recognize summer/winter time, but can always be switched back and works even if the power or the Internet fails.
The smart versions have their own buzz (“loudspeaker”) but how to integrate that into HA, I have no idea… Maybe it would be easier to install the full instructions for NSPanel in ESPHome, new Nextion display (yellow panel instead of red or black), load the complete finished tft on it, and then try to change any unwanted functions… In ESPHome you can “change” the sensors, i.e. instead of the “ntc sensor” install a DHT or DS18b ( i have both in my projects, one is for indoor, other for outdoor
Temperature, I don’t use the temperature from the weather service, but my own, I “stream” it from a Philips outdoor motion sensor that is on the doorstep (it shows the right temperature in the bedroom, whether I should put on long trousers after getting up , or the underpants are also enough to drink a coffee on the terrace… My structure in Editor is not complicated, I press on outside temperature, and thus I see “page06” with weather forecast Set heating, switch on/off, because for me it is done automatically by rules from Alexa (which was easier for me, like “Automaton in HA”), either the heating switches on (opens the heating control head for floors, or radiators + starts the gas boiler if there is not enough warm water in the pipes - >50°C - or if the room temperature is over > 28°C, Alexa switches on the air conditioner for

The only thing I really want to achieve is if I turn on something with Alexa, I want to see it on Nextion (like in your illustration, the little blue bar above the physical switches) and when I turn on something that currently works via MQTT, see it me in HA. So my expectations are minimal. But her performance is beautiful, especially the watering of flowers…

Betty-G · August 15, 2022, 7:27am

hello, i read through this “boiler thing” in your description, but…
You write in your demo that “round slider” is a timer.
Is it a translation error? Seems more like “thermostat setting” to me.
Question;
Please, do you have a solution for this? I have an “xflow” (that’s a value from a meter) in Nextion. How do I get it to change the value accordingly?

And: I formulated something else in my Arduino sketch for an Adafruit display:
}if (msvalue3 < {650
tft.setTextColor(GREEN, BLACK);
tft.println(“okay”);
}
}if (msvalue3 > {650
tft.setTextColor(RED, BLACK);
tft.println(“dry”);
}
how do i get that on a nextion?
or on esphome?