Outdoor illuminance template sensor

Code last updated: Nov 22, 2021


Status Update: Sept 24, 2020


Grab a copy of the template code below and start using it! The intent of this code is to give all of you (that aren’t comfortable with Python) the ability to customize the condition factors and even the sun factor in any way that works best for you.

The template code is performing very well! I started this as a personal challenge just to see if I could do it, and I did. I don’t plan on updating the code much in the future. Consider this your starting point!

By the way, I ran it side-by-side with https://github.com/nkm8/ha-illuminance and this was the result overnight! The reason for the slight difference in timing is ha-illuminance estimates dawn and dusk, while I am getting, what I assume is a more accurate time from @pnbruckner’s sun2 component. Due to this comparison I modified the template code to keep the lowest lx value at 10 instead of 0.

Screenshot 2020-09-24 074700

sensor.illuminance - template sensor

# pnbruckner's sensor component as a template.
# https://github.com/pnbruckner/ha-illuminance/blob/master/custom_components/illuminance/sensor.py
platform: template
sensors:
  illuminance:
    friendly_name: Outdoor Illuminance Educated Guessor
    icon_template: mdi:brightness-auto
    unit_of_measurement: lx
    value_template: |
      {%- set factors = namespace(condition='',sun='') %}

      {#- Retrieve the current condition and normalize the value #}
      {%- set current_condition = states("weather.santee") %}
      {%- set current_condition = current_condition|lower|replace("partly cloudy w/ ","")|replace("mostly cloudy w/ ","")|replace("freezing","")|replace("and","")|replace("-", " ")|replace("_", " ")|replace("(","")|replace(")","")|replace(" ", "") %}
      
      {#- Assign a seemingly arbitrary number to the condition factor #}
      {%- set condition_factors = {
        "10000": ("clear", "clearnight", "sunny", "windy", "exceptional"),
        "7500": ("partlycloudy", "partlysunny", "mostlysunny", "mostlyclear", "hazy", "hazysunshine", "intermittentclouds"),
        "2500": ("cloudy", "mostlycloudy"),
        "1000": ("fog", "rainy", "showers", "pouring", "snowy", "snowyheavy", "snowyrainy", "flurries", "chanceflurries", "chancerain", "chancesleet", "drearyovercast", "sleet"),
        "200": ("hail", "lightning", "tstorms")
      } %}
      {%- for factor in condition_factors if current_condition in condition_factors[factor] %}
        {%- set factors.condition = factor %}
      {%- endfor %}
      
      {#- Compute Sun Factor #}
      {%- set right_now = states.sensor.time.last_updated.timestamp() %}
      {%- set sunrise = states("sensor.sunrise") | as_timestamp %}
      {%- set sunrise_begin = states("sensor.dawn") | as_timestamp %}
      {%- set sunrise_end = sunrise + (40 * 60) %}
      {%- set sunset = states("sensor.sunset") | as_timestamp %}
      {%- set sunset_begin = sunset - (40 * 60) %}
      {%- set sunset_end = states("sensor.dusk") | as_timestamp %}
      {%- if sunrise_end < right_now and right_now < sunset_begin %}
        {%- set factors.sun = 1 %}
      {%- elif sunset_end < right_now or right_now < sunrise_begin %}
        {%- set factors.sun = 0 %}
      {%- elif right_now <= sunrise_end %}
        {%- set factors.sun = (right_now - sunrise_begin) / (60*60) %}
      {%- else %}
        {%- set factors.sun = (sunset_end - right_now) / (60*60) %}
      {%- endif %}
      {%- set factors.sun = 1 if factors.sun > 1 else factors.sun %}
      
      {# Take an educated guess #}
      {%- set illuminance = (factors.sun|float * factors.condition|float) | round %}
      {%- set illuminance = 10 if illuminance < 10 else illuminance %}
      {{ illuminance }}

sun2 sensor config

platform: sun2
monitored_conditions:
  - sunrise
  - sunset
  - dawn
  - dusk

Please see the CHANGELOG post below if you’d like to see any past changes.


Original Post

Poor @pnbruckner, he created a terrific outdoor illuminance sensor component. Unfortunately every time he adds support for a new weather component, support for that service get pulled for one reason or another.

I took a look at the code today and decided it would be a nice challenge to attempt to convert his Python code into a Home Assistant sensor template. I think I have succeeded. I haven’t had it running for more than a few hours, and I have already caught a few mistakes. I made use of @pnbruckner’s sun2 component to gather today’s sunrise and sunset values.

The nice thing about this is it is a template that we can modify to add support to the list of conditions for any weather provider. I’m using Accuweather as my primary weather provider, and I tried to create a fairly comprehensive list of conditions by referencing their Weather Icons API. I added most of their condition text values in the condition_factors dictionary. I also use some replacement filters in an attempt to make conditions be more consistent.

11 Likes

The timing of this couldn’t be better as the other one broke yesterday. So far it’s working great. thanks for this!

1 Like

CHANGELOG

Feb 5, 2022

Fix for the Optional: Multiple Weather Condition Sensors only. Accuweather is now returning “unavailable”, which was causing this code to return the wrong answer. With the code change below neither “unknown” or “unavailable” are accepted.

Replace

{%- for sensor in weather_sensors if states(sensor) != "unknown" and factors.current_condition == "" %}

with

{%- for sensor in weather_sensors if states(sensor) not in ["unknown","unavailable"] and factors.current_condition == "" %}

May 1, 2021 update

“Astral” error? Update the Sun2 Component.


Change #15

  1. Use now() instead of requiring setup of sensor.time per @DavidFW1960’s suggestion.
  2. Added comment encouraging users adjust condition_factors based on personal observations, as I have.

Change #14

Correction made to the Optional: Multiple Weather Condition Sensors. The code always returned the value of the last valid sensor instead of the first valid sensor.


Change #13

I mistakenly left a code fragment which was intended for the Optional: Multiple Weather Condition Sensors . Sorry about that. ( factors.current_condition should have just been current_condition .) The code is now correct in the first post.


CHANGE #12
A small change to the line that does normalizes the current_conditions variable (with lots of replace() filters. The |replace(" ", "") needs to be moved so it is the last replace on that line. I ran into a situation this morning where the condition was “mostly cloudy”, which there is no match for because there is supposed to be no spaces in the condition text.


CHANGE #11
Illuminance now does not go below 10 lx, just like the original component.


CHANGE #10
FIXED THE MATH! The code in the first post has been updated!


CHANGE #9
The entire code block has been replaced in the first post!

The morning’s slope was not correct due to mistaking sunrise_end with dusk. But, that change made it so the sun_factor never goes above 1.0, so my if sun_factor > 1 workaround is no longer necessary so it has been removed.

While discussing the code optimization with @123 I removed the unnecessary macros. Truthfully they were there to help wrap my brain around each step of the process. Now that I am getting a strong grasp on what the code needs to do, it was just making the code take longer to execute.

I also learned about namespace objects and that they are needed to smuggle a variable out of a for loop, without resorting to using a macro. So, I created a namespace named factors with 2 properties: condition and sun. factors.condition was necessary to get that value out of the for loop, while factors.sun was technically unnecessary, but I included it to make my brain happy. :wink:


CHANGE #8
Added sun2 sensor config to bottom of the original post.

platform: sun2
monitored_conditions:
  - daylight
  - night
  - elevation
  - sunrise
  - sunset
  - dawn
  - dusk

CHANGE #7
Corrected sunrise_end value. Was mistakenly assigned dusk before.

{%- set sunrise = as_timestamp(states("sensor.sunrise")) %}
{%- set sunrise_begin = as_timestamp(states("sensor.dawn")) %}
{%- set sunrise_end =  sunrise + (40 * 60) %}
{%- set sunset = as_timestamp(states("sensor.sunset")) %}
{%- set sunset_begin = sunset - (40 * 60) %}
{%- set sunset_end = as_timestamp(states("sensor.dusk")) %}

CHANGE #6
Updated the list of condition_factors based on Ecobee from nkm8’s ha-illuminance fork. GitHub - nkm8/ha-illuminance: Home Assistant Illuminance Sensor

{% set condition_factors = {
  "10000": ("clear", "clearnight", "sunny", "windy", "exceptional"),
  "7500": ("partlycloudy", "partlysunny", "mostlysunny", "mostlyclear", "hazy", "hazysunshine", "intermittentclouds"),
  "2500": ("cloudy", "mostlycloudy"),
  "1000": ("fog", "rainy", "showers", "snowy", "snowyheavy", "snowyrainy", "flurries", "chanceflurries", "chancerain", "chancesleet", "drearyovercast", "sleet"),
  "200": ("hail", "lightning", "tstorms")
} %}

CHANGE #5
I renamed the illuminance values to condition_factor. Review the code in the original post and you will find two changes:

  1. {% macro get_illuminance() %} is now {% macro get_condition_factor %}.
  2. {% set illuminance = get_illuminance() %} changed to {% set condition_factor = get_condition_factor() %}
    This change more accurately reflects the nature of the value, as it is NOT the illuminance value, but instead one factor in its calculation.

CHANGE #4
The sun_factor now caps out at 1 to mimic the functionality of ha-illuminance.

{% if sunrise_end < right_now and right_now < sunset_begin %}
  {%- set sun_factor = 1 %}
{% elif sunset_end < right_now or right_now < sunrise_begin %}
  {%- set sun_factor = 0 %}
{%- elif right_now <= sunrise_end -%}
  {%- set sun_factor = ((right_now - sunrise_begin) / (60*60*60)) * 10 %}
{%- else -%}
  {%- set sun_factor = ((sunset_end - right_now) / (60*60*60)) * 10 %}
{% endif %}

{%- if sun_factor|float < 1.0 %}
  {{ sun_factor }}
{%- else %}
  1
{% endif %}

CHANGE #3
At 10 am and my lx value with a clear condition was only 622! My gut is telling me the values are off by a factor of 10. So I am multiplying the time calculated values by 10, and my numbers are looking far better.

{% if sunrise_end < right_now and right_now < sunset_begin %}
  1
{% elif sunset_end < right_now or right_now < sunrise_begin %}
  0
{%- elif right_now <= sunrise_end -%}
  {{ ((right_now - sunrise_begin) / (60*60*60)) * 10 }}
{%- else -%}
  {{ ((sunset_end - right_now) / (60*60*60)) * 10 }}
{% endif %}

CHANGE #2
This next line is where I get the current weather condition text. My intention was to automatically failover between 3 separate weather integrations. But I just tested it with a nonexistent sensor and it didn’t work. To simplify I will just remove that in my example.

However, if you want to do this as well, here is the correct way to accomplish failover to add reliability to this sensor, try this.

{% if states("weather.accuweather") != "unknown" %}
  {% set current_condition = states("weather.accuweather") %}
{% elif states("sensor.openweathermap_condition") != "unknown" %}
  {% set current_condition = states("sensor.openweathermap_condition") %}
{% elif states("sensor.cc_climacell_weather_condition") != "unknown" %}
  {% set current_condition = states("sensor.cc_climacell_weather_condition") %}
{% endif %}
{% set current_condition = current_condition|lower|replace("partly cloudy w/ ","")|replace("mostly cloudy w/ ","")|replace("freezing","")|replace("and","")|replace(" ", "")|replace("-", " ")|replace("_", " ")|replace("(","")|replace(")","") %}

CHANGE #1
I have come to realize that a workaround is necessary to get the sensor to update (without resorting to a homeassistant.update_entity automation).

{% set FORCE_UPDATE_WORKAROUND = states("sensor.time") %}
3 Likes

You are welcome. Remember this is a work in progress. this morning is the first time I am getting to watch the chart climb. So I’ll see soon if I understand the code as well as I hope I do. :blush:

It has been overcast this morning, until about 10 minutes ago. Hopefull that chart will rise quickly once Accuweather realizes the sun is out.

My gut is telling me the values are off by a factor of 10. It’s 10 am and my lx value is still only 622!

As an experiment I am changing the sun_factor calculation to multiple the time of day factors by 10. I will keep you up to date as the day progresses.

If you have any insight, please share your thoughts on these calculations.

{% if sunrise_end < right_now and right_now < sunset_begin %}
  1
{% elif sunset_end < right_now or right_now < sunrise_begin %}
  0
{%- elif right_now <= sunrise_end -%}
  {{ ((right_now - sunrise_begin) / (60*60*60)) * 10 }}
{%- else -%}
  {{ ((sunset_end - right_now) / (60*60*60)) * 10 }}
{% endif %}

Watching progress with great interest. Keep up the good work

p.s. loving the friendly name

1 Like

Screenshot 2020-09-21 113605
The numbers are looking much more like I expected. I updated the code in the original code in the first post to multiple the sun_factor by 10.

1 Like

Hmm…Looking at the Phil’s original component his lx appears to have a hard limit of 10,000. At 12:40pm mine is 10,886.

I wonder if the calculations are off, or do I just need to limit the sun_factor to never go above 1?

Update: 1:30pm - I made the change to the code.

Continued from @Whiskey’s reply on Outdoor illuminance estimated from weather conditions - #116 by Whiskey

I have learned to do so much with templates over the last 2 years with Home Assistant! But, I am really happy with the automation/script choose and variable methods added to recent Home Assistant updates.

Please do use this. I am sharing this code in the hopes that someone like you will find it useful. I do think this has the potential to be a replacement for Phil’s terrific component – as long as the numbers don’t go so crazy that lights turn on in the middle of the night – with the advantage that guys like you and me can modify it to work with whatever components we want! :slight_smile:

1 Like

The latest updates to automations is awesome. The rate of development is fire right now.

I never knew about jinja macros. Can’t wait to rewrite some stuff a lot neater by implementing them.

I’ve just bought a BH1750FVI Lux sensor to measure actual lux levels in my apartment. Going to use it to calibrate my rest API cloudiness based Illuminance guesser sensor. I’m a little scared the hardware based lux sensor will grow from a calibration tool and become a permanent part of my system, making all this template guesstimating irrelevant!

Do you think my ‘current cloud cover percentage polled every 15 minutes’ idea will return more accurate and smoother results rather than five step values from icon/symbol data? Probably won’t notice much difference in the long run because the lights are working great as is.

If I can use your work in converting @pnbruckner’s python code to jinja then I can see me rolling all the templates into one sensor that spits out a brightness and temperature ready to be consumed by automations directly.

I will learn python one day!

All very cool.

Yeah, they referred to as functions in most other languages. The output of the macro is whatever you output (using {{ }}). You can also pass values into the macro. For example this macro receives a comma separated list and returns a sorted list ready for Alexa to speak.

{%- macro get_friendly_list(list) -%}
  {%- set comma = joiner(', ') -%}
  {%- for item in list|sort if list|length > 0 -%}
    {{ ' and ' if loop.last and not loop.first else comma() -}}
    {{ item|title }}
  {%- endfor -%}
{%- endmacro -%}

{%- set names = ["John", "Paul", "Ringo", "George"] %}
{{ get_friendly_list(names) }}
1 Like

Way to… lite a fire under me. :wink:

Looking at the history, the morning and daytime are working as I believe they should. However, the evening drops off an 10000 lx edge straight to zero. This obviously isn’t the desired outcome. So, I know there must be a logic problem if the sun_factor if statement.

Screenshot 2020-09-22 092333

P.S. I cheated and used a database editor to weed out all of the erroneous values I created with mistakes throughout the day. That’s why that chart looks cleaner than it should. :blush:

FWIW, macros are advantageous because they reduce code-length when they are called more than once. They are only called once in your Template Sensor so that advantage is lost. In fact, they serve to increase code-length (due to the macro’s delimiting statements and the macro call itself).

Here’s a shorter version of your code (without macros). Please note, it is untested because I don’t use the sun2 custom component.

platform: template
sensors:
  illuminance:
    friendly_name: Outdoor Illuminance Educated Guessor
    icon_template: mdi:brightness-auto
    unit_of_measurement: lx
    value_template: >
      {% set condition_factors = {
          "10000": ("clear", "clearnight", "sunny", "windy", "exceptional"),
          "7500": ("partlycloudy", "partlysunny", "mostlysunny", "mostlyclear", "hazy", "hazysunshine", "intermittentclouds"),
          "2500": ("cloudy", "mostlycloudy"),
          "1000": ("fog", "rainy", "showers", "snowy", "snowyheavy", "snowyrainy", "flurries", "chanceflurries", "chancerain", "chancesleet", "drearyovercast", "sleet"),
          "200": ("hail", "lightning", "tstorms") } %}
      {% set current_condition = states("sensor.openweathermap_condition") %}
      {% set ns = namespace(condition_factor='') %}
      {%- for factor in condition_factors if current_condition in condition_factors[factor] -%}
          {% set ns.condition_factor = factor }}
      {%- endfor %}

      {%- set right_now = states.sensor.time.last_updated.timestamp() %}
      {%- set sunrise = as_timestamp(states("sensor.sunrise")) %}
      {%- set sunrise_begin = as_timestamp(states("sensor.dawn")) %}
      {%- set sunrise_end =  sunrise + (40 * 60) %}
      {%- set sunset = as_timestamp(states("sensor.sunset")) %}
      {%- set sunset_begin = sunset - (40 * 60) %}
      {%- set sunset_end = as_timestamp(states("sensor.dusk")) %}

      {% if sunrise_end < right_now < sunset_begin %}
        {%- set sun_factor = 1 %}
      {% elif sunset_end < right_now or right_now < sunrise_begin %}
        {%- set sun_factor = 0 %}
      {%- elif right_now <= sunrise_end -%}
        {%- set sun_factor = ((right_now - sunrise_begin) / (60*60*60)) * 10 %}
      {%- else -%}
        {%- set sun_factor = ((sunset_end - right_now) / (60*60*60)) * 10 %}
      {% endif %}
      {%- set sun_factor = sun_factor if sun_factor|float < 1.0 else 1 %}
    
      {{ ns.condition_factor }} {{ (sun_factor|float * ns.condition_factor|float) | round }}
1 Like

I found a mistake that make this more accurate. I accidentally assigned the dusk sensor to sunrise_end instead of sunset_end. I am updating the code in the first post.

{%- set sunrise = as_timestamp(states("sensor.sunrise")) %}
{%- set sunrise_begin = as_timestamp(states("sensor.dawn")) %}
{%- set sunrise_end =  sunrise + (40 * 60) %}
{%- set sunset = as_timestamp(states("sensor.sunset")) %}
{%- set sunset_begin = sunset - (40 * 60) %}
{%- set sunset_end = as_timestamp(states("sensor.dusk")) %}

I agree, however I often use functions to break up the functionality of a complex bit of code into manageable chunks. That works best for my brain while I am figuring out a new process.

I’ve learned a lot from your code optimization replies over the years! Thank you! I am unsure of the use of namespace though. I will have to look into that.

FWIW, because your template uses the current time to perform a calculation, you can dispense with this line:

{% set FORCE_UPDATE_WORKAROUND = states("sensor.time") %}

and incorporate sensor.time directly in the time calculation. Simply replace this line:

{%- set right_now = as_timestamp(now()) %}

with this:

{%- set right_now = states.sensor.time.last_updated.timestamp() %}

The subtle difference between the two is that the resolution of the first one is in seconds whereas the second one is in minutes. However, for the purposes of this template, that’s sufficient.

2 Likes

Sure but there are only two so-called ‘chunks’ here. Effectively you’ve done this (pseudo-code):

macro add()
  return 2+2
macro subtract()
  return 2-2

x = add()
y = subtract()

{{ x }}
{{ y }}

I can see how that’s advantageous if there’s a need to call add or subtract several times but, in this case, there isn’t. Without macros it’s just this (with no loss of legibility):

x = 2+2
y = 2-2

{{ x }}
{{ y }}

A variable used within a for-loop has a limited scope. Changes to its value inside the loop aren’t accessible outside the loop.

You can easily demonstrate it for yourself. Notice the final value of x inside the loop is 4 but outside the loop it’s 0.

{% set x = 0 %}
{% for i in range(1,5) %}
  {% set x = i %}
  i={{ i }}, x={{x}}
{% endfor %}

The value of x is: {{ x }}

namespace allows us to define variables with a broader scope.

{% set ns = namespace(x = 0) %}
{% for i in range(1,5) %}
  {% set ns.x = i %}
  i={{ i }}, x={{ns.x}}
{% endfor %}

The value of x is: {{ ns.x }}

In this version, the value of ns.x outside the loop is the same as inside the loop.

2 Likes

Thank you for that! I’ve felt like I saw there was a more elegant way to do that… I just wasn’t finding it.

I also see that namespaces are necessary to get a value out, for some reason, when you stick and if statement on the same line as your for statement. :man_shrugging: But, I’ve got my head around them now… and this still adds clarity while not really making my variable names longer.

{% set factors = namespace(condition='',sun='') %}
...
{%- for factor in condition_factors if current_condition in condition_factors[factor] %}
  {%- set factors.condition = factor %}
{%- endfor %}
...
{% if sunrise_end < right_now and right_now < sunset_begin %}
  {%- set factors.sun = 1 %}
...
{{ (factors.sun|float * factors.condition|float) | round }}

I suspected something along those lines… OOOOOOhhhh, I didn’t have that problem before because I was returning that value from a macro directly! So, I wasn’t having that problem.

I wonder if I have unconsciously resorted to macros more often than seems necessary without realizing this! :grin:

Oh and as far as that goes. I agree… and I sometimes, not always optimize my code in the end. In this case I was trying to convert Python code doing some math I didn’t understand at first.

Thank you.

1 Like