Outdoor illuminance template sensor

[123 (Taras)]

There’s probably no pressing need to make sun a namespace variable (because its value isn’t calculated within any for-loop) but you can certainly do that for, I guess, the sake of neatness.

@BrianHanifin

I’ve felt like I saw there was a more elegant way to do that

Just keep in mind that they are NOT precisely the same thing. Paste this into the Template Editor and you’ll see that the one based on now() updates every second whereas the other one, based on sensor.time is every minute.

{{ now().timestamp() }}
{{ states.sensor.time.last_updated.timestamp() }}

{{ now().timestamp() | timestamp_local() }}
{{ states.sensor.time.last_updated.timestamp() | timestamp_local() }}

For many time calculations, every minute is ‘good enough’. For calculations that are comparing times down to the second, that’s not ‘good enough’.

[BrianHanifin (Brian)]

@123

There’s probably no pressing need to make sun a namespace variable (because its value isn’t calculated within any for-loop) but you can certainly do that for, I guess, the sake of neatness.

I knew it was technically unnecessary. But there is enough chaos going on in my brain already. It helps me to find ways to keep code neat.

@123

Just keep in mind that they are NOT precisely the same thing. Paste this into the Template Editor and you’ll see that the one based on now() updates every second whereas the other one, based on sensor.time is every minute .

That’s actually better. I don’t want this to update too often.

[123 (Taras)]

now() is a function, not an entity, so Home Assistant doesn’t assign a listener to it. Therefore there was never any chance of the Template Sensor being evaluated every second.

You can see that in the Template Editor which now (in 0.115) reports which entities are being monitored (if any):

[BrianHanifin (Brian)]

OK. So here is how the sensor ended the day. It stays at 10000 lx until 6:06:05 pm. At 6:06:06 pm the value immediately falls to 1795 lx. Does this seem right to anyone that has used the component before? I was expecting a gradual dropoff, I’m no expert on this.

I created this test code in the Template Developer Tool. It allows me to manipulate the right_now time so I can see how the code changes the values at a certain time of day. It is more or less performing as I expected.

Test code

{%- set sunrise = states("sensor.sunrise") | as_timestamp %}
{%- set sunrise_begin = states("sensor.dawn") | as_timestamp %}
{%- set sunrise_end = sunrise + (40 * 60) %}
{%- set sunset = states("sensor.sunset") | as_timestamp %}
{%- set sunset_begin = sunset - (40 * 60) %}
{%- set sunset_end = states("sensor.dusk") | as_timestamp %}

{% set right_now = now().replace(hour=18,minute=06,second=06) | as_timestamp %}
right_now: {{ right_now | timestamp_custom("%I:%M:%S %p") }}

sunrise_begin: {{ sunrise_begin | timestamp_custom("%I:%M:%S %p") }}
sunrise: {{ sunrise | timestamp_custom("%I:%M:%S %p") }}
sunrise_end: {{ sunrise_end | timestamp_custom("%I:%M:%S %p") }}
sunset_begin: {{ sunset_begin | timestamp_custom("%I:%M:%S %p") }}
sunset: {{ sunset | timestamp_custom("%I:%M:%S %p") }}
sunset_end: {{ sunset_end | timestamp_custom("%I:%M:%S %p") }}

if {{ sunrise_end < right_now }} and {{ right_now < sunset_begin }}
  factors.sun = 1
elif {{ sunset_end < right_now }} or {{ right_now < sunrise_begin }}
  factors.sun = 0
elif {{ right_now <= sunrise_end }}
  factors.sun = {{ ((right_now - sunrise_begin) / (60*60*60)) * 10 }}
else
  factors.sun = {{ ((sunset_end - right_now) / (60*60*60)) * 10 }}

Output

right_now: 06:06:06 PM

sunrise_begin: 06:10:46 AM
sunrise: 06:35:24 AM
sunrise_end: 07:15:24 AM
sunset_begin: 06:06:06 PM
sunset: 06:46:06 PM
sunset_end: 07:10:44 PM

if True and False
  factors.sun = 1
elif False or False
  factors.sun = 0
elif False
  factors.sun = 1.9870406117152284
else
  factors.sun = 0.1795334623588456

Here is the current Python code starting at line 300 from the ha-illuminance component. This is the key to getting the values correct, and I am not 100% sure I converted the if statements correctly. Can anyone more familiar with Python confirm my translation is correct?

if sunrise_end < now < sunset_begin:
   # Daytime
  return 1
if now < sunrise_begin or sunset_end < now:
  # Nighttime
  return 0
if now <= sunrise_end:
  # Sunrise
  return (now-sunrise_begin).total_seconds() / (60*60)
# Sunset
return (sunset_end-now).total_seconds() / (60*60)

My template translation

{%- if sunrise_end < right_now and right_now < sunset_begin %}
  {%- set factors.sun = 1 %}
{%- elif sunset_end < right_now or right_now < sunrise_begin %}
  {%- set factors.sun = 0 %}
{%- elif right_now <= sunrise_end %}
  {%- set factors.sun = ((right_now - sunrise_begin) / (60*60*60)) * 10 %}
{%- else %}
  {%- set factors.sun = ((sunset_end - right_now) / (60*60*60)) * 10 %}
{%- endif %}

[BrianHanifin (Brian)]

So, using the above template and a database editor, I modified the mornings graph to simulate how the current code would ramp up. Here is a screenshot of roughly what it would have looked like.

So it looks like lx starts climbing at sunrise_begin and abruptly ends at sunrise_end. Then makes and abrupt jump straight to 10000. It seems odd, but unless I am misinterpreting the Python code, I think my template is working the same as the Python code.

[BrianHanifin (Brian)]

There is a math translation error. The results are incorrect currently. I realized this morning that I could probably use @craigb’s pyscript component to evaluate the original code and this has been incredibly helpful!

I was able to set both my template script and the pyscript code to the same time of day to compare their sun factor values. With both set 1 second before the sunrise_end time (7:16:02 am) the Python code output 0.9997222222222222 while my template produced 0.16665227643869543.

Python/pyscript

now = datetime.datetime(2020, 9, 23, 7, 16, 2)

Home Assistant Template
A note on hour=(7 +7): to make it easier for my brain I included the timezone offset to set the UTC time to match my local, California time.

{%- set test_time = states.sensor.time.last_updated.replace(hour=(7 +7),minute=16,second=2) %}
{%- set right_now = test_time.timestamp() %}

My brain is overheated. I’m going to park it for a while. But now that I know where the problem is, I should be able to figure it out.

[BrianHanifin (Brian)]

@BrianHanifin

I was able to set both my template script and the pyscript code to the same time of day to compare their sun factor values. With both set 1 second before the sunrise_end time (7:16:02 am) the Python code output 0.9997222222222222 while my template produced 0.16665227643869543 .

SUCCESS!!!

Code updated (see first post)
This is exactly what I needed. The calculations now match the custom_component. The math was messed up, but I couldn’t see how until I could see the results of the python math.

Honestly I credit explaining the problems on this thread to helping me resolve this! I hope you don’t mind.

[123 (Taras)]

What kind of dark clouds from hell appeared in your neighborhood to cause illuminance to drop from 10000 to 1000?

[BrianHanifin (Brian)]

Haha! OpenWeatherMap reported “Fog”. I think getting a weather source you trust is the trickiest thing with this whole system. But, that is why I wanted to create a template version. This way ANYBODY can modify the list of condition_factors to match their weather condition provider. Although, I tried very hard to make mine fairly universal through data normalization.

[BrianHanifin (Brian)]

Update: Sept 24, 2020

Grab a copy of the template code from the original post and start using it! The intent of this code is to give all of you (that aren’t comfortable with Python) the ability to customize the condition factors and even the sun factor in any way that works best for you.

The template code is performing very well! I started this as a personal challenge just to see if I could do it, and I did. I don’t plan on updating the code much in the future. Consider this your starting point!

Here is a screenshot how the template performed in the evening and this morning as compared to nkm8’s fork of ha-illuminance. The reason for the slight difference in timing is ha-illuminance estimates dawn and dusk, while I am getting, what I assume is a more accurate time from @pnbruckner’s sun2 component. Due to this comparison I modified the template code to keep the lowest lx value at 10 instead of 0.

[BrianHanifin (Brian)]

Optional: Multiple Weather Condition Sensors
If your primary weather source goes “unavailable” sometimes, you can modify a few lines of code to add one or more backup sensors.

current_condition: original lines

{%- set factors = namespace(condition='',sun='') %}

{#- Retrieve the current condition and normalize the value #}
{%- set current_condition = states("weather.accuweather") %}
{%- set current_condition = current_condition|lower|replace("partly cloudy w/ ","")|replace("mostly cloudy w/ ","")|replace("freezing","")|replace("and","")|replace(" ", "")|replace("-", " ")|replace("_", " ")|replace("(","")|replace(")","") %}

current_condition: new lines (with 2 backup condition sensors)

{%- set factors = namespace(condition='',sun='',current_condition='') %}

{#- Retrieve the current condition and normalize the value #}
{%- set weather_sensors = [
  "weather.accuweather",
  "sensor.openweathermap_condition",
  "sensor.cc_climacell_weather_condition"
] %}
{%- for sensor in weather_sensors if states(sensor) not in ["unknown","unavailable"] and factors.current_condition == "" %}
  {%- set factors.current_condition = states(sensor) %}
{%- endfor %}
{%- set current_condition = factors.current_condition|lower|replace("partly cloudy w/ ","")|replace("mostly cloudy w/ ","")|replace("freezing","")|replace("and","")|replace(" ", "")|replace("-", " ")|replace("_", " ")|replace("(","")|replace(")","") %}

[BrianHanifin (Brian)]

A small change to the line that does normalizes the current_conditions variable (with lots of replace() filters. The |replace(" ", "") needs to be moved so it is the last replace on that line. I ran into a situation this morning where the condition was “mostly cloudy”, instead of “mostlycloudy” like I expected.

[larryw (Larry)]

@BrianHanifin

{%- set current_condition = factors.current_condition|lower|replace("partly cloudy w/ ","")|replace("mostly cloudy w/ ","")|replace("freezing","")|replace("and","")|replace("-", " ")|replace("_", " ")|replace("(","")|replace(")","")|replace(" ", "") %}

I believe that this line of code is incorrect, if I comment it out the template runs, but otherwise it always returns a value of 10. I believe the issue is after the assignment you cannot reference factors.current_condition as that does not exist. It just needs to be current_condition = current_condition…
So the corrected line is:

{%- set current_condition = current_condition|lower|replace("partly cloudy w/ ","")|replace("mostly cloudy w/ ","")|replace("freezing","")|replace("and","")|replace("-", " ")|replace("_", " ")|replace("(","")|replace(")","")|replace(" ", "") %}

Thanks!

[BrianHanifin (Brian)]

Oops! I just realized what the problem was. Here is the correct line of code:

{%- set current_condition = current_condition|lower|replace("partly cloudy w/ ","")|replace("mostly cloudy w/ ","")|replace("freezing","")|replace("and","")|replace("-", " ")|replace("_", " ")|replace("(","")|replace(")","")|replace(" ", "") %}

I mistakenly left a code fragment which was intended for the Optional: Multiple Weather Condition Sensors. Sorry about that. (factors.current_condition should have just been current_condition.)

{%- set current_condition = factors.current_condition|lower|replace("partly cloudy w/ ","")|replace("mostly cloudy w/ ","")|replace("freezing","")|replace("and","")|replace("-", " ")|replace("_", " ")|replace("(","")|replace(")","")|replace(" ", "") %}

[pnbruckner (Phil)]

@BrianHanifin

Poor @pnbruckner, he created a terrific outdoor illuminance sensor component. Unfortunately every time he adds support for a new weather component, support for that service get pulled for one reason or another.

Yep, HA has been, and continues to be, a moving target. FYI, just released updates to my custom integration to deal with changes through 0.117. I don’t use it anymore myself (since I got a light sensor a while ago) so can’t be sure it’s 100% right, but it does appear to work again (and at least doesn’t crash anymore.) Having said that seems like you’ve gone beyond what I did anyway.

[DavidFW1960 (David)]

@BrianHanifin

{%- set right_now = states.sensor.time.last_updated.timestamp() %}

For some reason my HA didn’t like that so I used:

          {#- Compute Sun Factor #}
          {%- set right_now = now() | as_timestamp %}

instead.

[DavidFW1960 (David)]

Also @BrianHanifin you seem to have updated to:

{%- elif right_now <= sunrise_end %}
  {%- set factors.sun = ((right_now - sunrise_begin) / (60*60*60)) * 10 %}
{%- else %}
  {%- set factors.sun = ((sunset_end - right_now) / (60*60*60)) * 10 %}
{%- endif %}

But the original post doesn’t reflect that. Can you confirm the code in Post 1 is actually correct?
It shows /(6060) not (6060*60))*10
Also I have to use now() as the other function doesn’t work and gives log errors

[grashugger (gras)]

I had problems with

{%- set right_now = states.sensor.time.last_updated.timestamp() %}

at first too, because I apparently did not have a time sensor in my config yet. I updated to include the following in sensors.yaml, which fixed the problem for me. (@BrianHanifin, maybe update the OP to reflect this dependency)

- platform: time_date
  display_options:
    - 'time'

[DavidFW1960 (David)]

Is there a reason to not use now()?

[grashugger (gras)]

I don’t believe there is.