This will outline my design for heating automation. I am a software engineer and a tinkerer and have always wanted to automate bits of my home. Once I had a few sensors on a Raspberry PI to create a weather station as a “mickey mouse” project I looked straight at heating control as the defacto home automation application.
I decided to go completely “clean room” on it. Build it up from scratch, not accept or limit myself to any particular convention or standard and not even look at what others were doing. Just starting from basic questions about how I would like to automate my heating and building it up.
In this way there are currently no immediate integration with HAss or OH or any other high level automation system. But I’m keen to see what can be done here.
Basic concepts:
-
Data - anything that might be of use in the system. These come from places like:
- Sensor readings
- Device states
- etc.
- Example: Heating Room Temp: 20.56*C
- Example: Main Heating: ON
- Example: Presence Paul: HOME
-
Demands - Demands are how the system signals that something is required. It provides a decoupling point between what is asking, what it is asking for and how the system can full-fill, or ignore that demand. Demands are:
- Timely - The have a timestamp when raised.
- Expire - They have a life expectancy after which they are no longer effective.
- Discretionary - The don’t have to be served or acknowledged.
- Expiry is enforced for automatics, overridden for manual intervention.
-
Schedulers, Schedules and Conditions are the main automated mechanisms to raise demands given a set of conditions. Schedules are competing, layered and many can and will be active at anyone time.
- Schedulers fetch data and run a set of Schedules in order.
- Schedules - A basic schedule is just a set of conditions, if true it raises a demand for something. More advanced Schedules can do much more including raising multiple demands and monitoring multiple zones.
- Condition - A condition accepts data and returns true or false when evaluated. Examples are, TimeOfDay, DayOfWeek, Presence, TemperatureMinimum
-
Controllers. Controllers are the integration points for controlling actual hardware. Often and especially in the case of heating there are safety stipulations and “appropriate use” mechanics and so these are contained close to the device in a controller specific to that device. Obvious examples might be “HeatingController” or “RadiatorController” - These are good candidates for replacing with HA.
-
Distribution and Isolation of concern - The system is mostly, except where I was being lazy, fully distributed. Data is stored in one place, Demands in another, Schedulers (multiple if you want) can be run anywhere you want, controllers can run anywhere you want. You can run it all on one host and use “localhost” if you wish or spread it out all over the place if you want. Each component does it’s job and trusts that other components will do theirs without caring how they do it.
-
Transient state - The acid test of this is to switch any component off and on again and see how long it takes to recover. This applies to any component or the whole system. It has to recover promptly to the correct state. To that end nothing is “stored”, nothing is “persisted” all logic is re-run periodically. Sensors send data every 5 seconds, schedules run every 30 seconds, demands are polled every minute. So it will automatically recover to the state it should be in once they run again in order. (I may have to persist dynamic configuration going forward).
-
Fail safe - If any component fails, stack dumps and exits, the heating should never be left running indefinitely but timeout and return to a safe state. Other features include locking out excessive commands at the device level, for example, anti-short-cycle for a boiler or leaving a NC radiator valve open constantly.
-
Transparent to manual intervention. “I’m sorry, I cannot let you do that Dave” is not an option. Devices should retain as much of their existing manual functionality as possible. Demand, Schedules, Conditions will just have to accept that a human may have changed some state in the system without consulting them. You cannot assume something is stil as the automation left it last.
-
Avoiding state machines - or moving them as close to the real world state as possible.
Here is an overview diagram:
The term “bulletin board” is used to describe incredibly lightweight storage used. Any data published to the bulletin board is available for all clients to read. There is no “delete”. If data doesn’t exist before it’s published it is created, if it does exist it is overwritten. Data has a timestamp but it is up to the client of the data to choose what “too old” is. Demands have a timestamp AND an expiry and in this case it is up to the publisher to decide how old the demand is allowed to be and still be effective. Without “delete”, things just go stale and time out and become ineffective and are ignored.
In the current implementation in Python, Data hub and Demand hub share a process. Scheduler and Demand Processor share a process. So excluding sensors there are currently 4 processes.
Schedules:
This shows a basic overview of how schedules work. Note the conditions are not limited to the intuitive “Time of day” etc. But can very easily trigger on any published data such as Presence or whether another device is ON or OFF.
The MultiroomTemperatureSchedule is probably the most used Schedule as it single handedly encapsulates multi-zone temperatures, presence and standard time scheduling in one place. It consumes from a simple Python Dictionary of target temps for various modes, there are several of these schedules with different conditions and different target modes or profiles, such as “Eco” or “Comfort” etc.
Multiple schedules will most likely be active. When I get up in the morning I have:
MaintenanceSchedule checking the temp is about 6C everywhere in the house and above 1C in the garage.
RoutineWeekDayMorningSchedule checking the temps in various rooms are meeting the “eco” target temps.
PresenceSchedule checking the temps in various rooms are meeting the “comfort” target temps.
The fact that more than one of these may raise a demand for heating is all sorted out in the Demand Processor.
Integration points:
Not a lot of these are currently very useful for HA intergartion. Most are raw sockets that send/receive JSON payloads. The REST end points are not really REST and more HTTP CGI. This is where I plan to do a lot of work leveraging better standards such as RESTful and maybe MQTT.
(Config hub is shown here as a future idea on how to manage dynamic configuration, encapsulating it into another network service)
An example message flow might look something like this:
This is close to what I have presently as I have no yet got automated radiator valves, so the while the Demand processor receives a demand for “livingroom heating”, all it can do is emit a demand for Main heating.
This is an example flow if I had automated radiator valves and shows how multiple zone demanding should works.
Given the latest sensor reading for livingRoom and office their temps are below target, based on the config for this schedule being “comfort” and the target temps for comfort being “20C”. The schedule raises demands for those zones.
You should also note the cool feature of RRD data logging. Regardless of what you send to the Data Hub it will create or update an RRD for that value and log it once a minute. No configuration required.
The Demand processor does the logic to determine how to satisfy the incoming demands. In this case I imagined it would raise demands for the various device states. Controllers (unshown) would receive those demands and command the devices (if required and if appropriate).
Logic for things like “Normally Closed, Normally Open” would be handled in the Controller. The Demand Processor raising the demand just wants the radiator open it doesn’t care how the controller does it.
A lot of the design comes from what I call “Scenario Driven Development”. It’s about thinking through scenarios and dry running (usually in my head) through scenarios to evaluate solutions and if they work or not.
In operation a demand is currently raised by the automatics to expire in 5 minutes. This means that if the computer running all or part of the operation dies or falls off the planet the heating will switch off in 5 minutes and all valves will return to their default state. This logic stretches down into the switch which controls the boiler relay, although not fully implemented with timeouts there yet (an open TODO for safety).
No schedules currently exist that demand the heating to be OFF. This is possible, but there isn’t a need in normal operation. Once the conditions that cause a demand to be raised are no longer true the existing demand will simply expire. While they are still true the demand will be refreshed periodically.
Demand Expiry cannot be overridden by automatics. If I manually publish a demand for 1 hours heating, then a schedule is forbidden from publishing that same demand until it expires or until it’s expiry would be ahead of the existing demand’s. This allows manual intervention overrides for both OFF and ON.
Demand Expiry can be overridden by manual intervention. A button used to command heating for 1 hour will not bother checking if a demand exists, it wins. “Sorry Hal.”
Hacks:
Presence state is currently published into the Demand hub as if it were a demand. This allows it to take advantage of the expiration and so I don’t need to worry about setting the presence as “AWAY”, I just let the Presence HOME expire.
Lighting Controller. This was hooked up to control group lights using the same system. SOnOff S20s or the RF Bridge via MQTT publish demands for these lights and the lighting controller switches them on. It has revealed that polling demand based control has draw backs for things you would expect to be have “realtime” response. So the Lighting Controller polls the demands once a second, creating a load of network noise, but can still result in a seconds delay between hitting the switch and the lights coming on.
Home Assistant:
Obviously my heating system was not designed with HA (or any other) system in mind. You could certainly integrate as is with Scripts running “netcat” to inject JSON onto bulletin boards and collect JSON back from them. You can control the devices with HTTP.
I realise a sore point will be the Data Hub. It kinda ties things together and it is where all my sensors publish to. They publish a JSON blob like:
{ “livingRoom”: { “key”:“livingRoom”, “shortName”:“LR”, “name”:“Living Room”, “units”:"*C", “value”: 19.56, “type”:“float” }
So you “could” create an integration which dumps all of your sensor values from HA to the data hub, or, an MQTT handler (not written yet) could subscribe and/or discover your current sensors and publish them for you.
Control of the system could be done by publishing and reading demands.
The heating controller (and lighting controller or any other) seem like a good candidate to just move to HA. I imagine an integration polling demands and a JSONPath expression to extract the one you want and send the signals to the switches/relays.
You will notice the lack of what I call half-smarts. I don’t like radiator valves which turn themselves on or off although I realise these exist. I have open TODOs for migrating some meta info into demands such as what Schedule raised the demand in the first place and where appropriate what target temperature was it trying to achieve. This could be used to push a set-point temp to an automated radiator valve.
Other TODOs for the next version is a predictive scheduler. A predictive scheduler will take a normal scheduler reference and run it in the future to ask the question “In Half an hour, would the heating need to be on?”, if that returns true, then the demands it would raise in the future can be checked and if appropriate the heating enabled early in order to reach the target temp in time. This is predictive or what I can “pre-ramping” the heating to achieve a temperature at a future time. By wrapping the whole scheduler in this mechanism means it support the same things as the actual Schedule, just running in the future.
What is missing.
UIs.
Dynamic config, such as altering the target temperature and persistence of.
Pretty displays
Proper standards at integration points.
Code - it’s a bit messy:
One final word. This is not the same kind of project as HA. I have never had intentions of making this pre-canned and ready to configure for anyone or everyone as a public project. A lot of it is simply hard-coded for my use. This keeps things very simple, but it means if you wanted to use the system you will need to be able to roll up your sleaves and get into editing some Python and/or C code.
