Outdoor illuminance estimated from weather conditions

Are you using Custom Updater 3.1.8? If so, then can’t explain. If not, then that might explain it.

EDIT: Unless you tell it to check for updates it only does that on its own once a day. That could also explain why it was still seeing 1.0.0.

maybe that. still thought it peculiar it saw 1.0.0 as an update to 2.0.0b5. Unless betas don’t count?

It doesn’t compare versions. It’s pretty simple – it only knows what you have and what is available at the source. If they’re different then it shows it as having an update which, to me, is the way it should be.

ok, I’ve worked it out.

please allow me to ask: the sensor illuminance shows 10 lx when dark, is that correct? Im not sure about the true lux numbers, but my philips light sensors go down to 1, and have many settings between 1 and 10.

lx is calculated as follows: lx = round(float(10**((lightlevel-1)/10000)), 2)

this is what Philips uses for their scale:

#Philips Hue definition                 Lux               MeasuredValue
#Overcast moonless night sky              0.0001                  0
#Outdoor: Bright moonlight                1                       1
#Home: Night light                        2                    3000
#Home: Dimmed light                      10                   10000
#Home: ‘Cosy’ living room                50                   17000
#Home: ‘Normal’ non-task light          150                   22000
#Home: Working / reading                350                   25500
#Home: Inside daylight                  700                   28500
#Home: Maximum to avoid glare          2000                   33000
#Outdoor: Clear daylight            > 10000                 > 40000
#Outdoor: direct sunlight            120000                   51000

and this is the sensitivity in home now:

42

while your yr sensor shows:

23

supposed to be darker outside then inside… is this a factor 10 off somehow?

It’s just a rough estimate. It’s not intended to be anywhere near as accurate as a light sensor. It only has a few levels: 10, 200, 1000, 2500, 7500 & 10000. And during an hour around sunrise and sunset it scales that value. It’s an algorithm that someone else implemented in Samsung Smartthings that I used when I used ST, and borrowed when I moved to HA. It’s been good enough for me to drive light simulations when we’re not home. It’s not intended for anything but that.

Ok I understand, thanks for explaining. Should have read your documentation (…) better.
With all these weather components, there must be one with a true measured light level? Seems too silly to not exist.

for reference:

http://www.domoticz.com/wiki/Real-time_solar_data_without_any_hardware_sensor_:_azimuth,_Altitude,_Lux_sensor

another(@Kobold 's) sun.py using the above: https://gist.github.com/anonymous/9dde1a5fadef4178bc288939176ee0d5
(note: also creating attribute daylight, but with another meaning: light level, vs amount of time from today’s sunrise to today’s sunset (in seconds) )

Maybe you would consider adding an extra attribute light level too to your enhanced sun CC?

and a thread here dedicated to the subject: Virtual light sensor

I’m now using this with DarkSky and today (now) I have a situation where Illuminance is giving

image

but the sun is showing

image

and the Dark Sky icon is

image

and Dark Sky cloud coverage is

image

Is this correct?
(Or maybe I have something configured wrongly - always an option!)

For the record here is todays Illuminance graph.

EDIT: While I was writing this it has been going down as expected so maybe this is all expected but if that is the case then I think the times today when it went down to 1000 may be suspect as it is a lot darker now than it was at any time during today. Despite the weather we’ve had!

Remember, this component can not look out your window to see how bright or dark it is! :wink:

It’s simply using the reported weather condition summary and looking that up in a table of estimated values, and around sunrise and sunset it ramps the value up and down. That’s it. No magic. As I said, I got this algorithm from an implementation someone else made that I was using in SmartThings. I don’t claim it’s perfect, or even very good. I’m sure there are better estimates out there – @Mariusthvdb pointed out a few. But it’s been good enough for my automation needs, at least with WU as the data source. I’ve been noticing that YR and Dark Sky don’t often agree with WU, or each other. So ultimately, this is limited by the weather data source. The more accurate/timely it is the better, and vice versa.

At this time I’m not looking to make this a better, more accurate estimation of outside light level. I appreciate the feedback and suggestions, but that’s not on my to-do list. I’ve bookmarked the suggestions and at some point down the road I may try some improvements.

So, bottom line, what you describe is kind of what I’d expect.

No problem., No criticism was meant, I am very appreciative of all your work not just on your custom components but also in answering questions here.

It was as you say, only meant as an observation in case you wanted to act on it but there is definitely no expectation on my part that you do.

Thanks again.

1 Like

fwiw, my ‘local’ weather component Buienradar, is horrifically slow at boot, causing all kinds of timing issues. And when all has finally been initiated, I see big differences between the components Yr, Yahoo, Buienradar and Open weather map.

Even on Temp or windspeed. Its amazing to experience that, and makes one wonder about the reliability of things.

To keeps things short: ultimately, a hardware light sensor is the only reliable source for outdoor illuminance i am afraid. Pointing my indoor sensors to the outside works fine already, and the soon to be released outdoor sensor for my Hue system should offer final answers… I hope.

Agree 100%. I’m just too cheap to go buy one. :wink:

2 Likes

I just upgraded my HASS install to 0.85.0 and I’m getting this error now:

Thu Jan 10 2019 20:30:54 GMT-0500 (Eastern Standard Time)

Error loading custom_components.sensor.illuminance. Make sure all dependencies are installed

Traceback (most recent call last): 
File "/usr/src/app/homeassistant/loader.py", line 92, in get_component 
module = importlib.import_module(path) 
File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module 
return _bootstrap._gcd_import(name[level:], package, level) 
File "<frozen importlib._bootstrap>", line 994, in _gcd_import 
File "<frozen importlib._bootstrap>", line 971, in _find_and_load 
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked 
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked 
File "<frozen importlib._bootstrap_external>", line 678, in exec_module 
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed 
File "/config/custom_components/sensor/illuminance.py", line 19, in <module> 
from homeassistant.components.sensor.darksky import ( 
ImportError: cannot import name 'CONF_ATTRIBUTION'

Anyone else getting this? Is this because they changed something in HASS?

Yes, this has been noticed and a PR has already been submitted. See Issue #83 and PR #84.

The root cause is a change in the darksky sensor platform.

Awesome, thanks. When I checked earlier on your git I didn’t see the issue, but I didn’t check the PR. Thanks for the fast response.

Released 2.0.1.

Fixes bug caused by change in Dark Sky Sensor in HA release 0.85. Now works with new HA version, as well as older versions.

Is there a way to force a sample, even if the illuminance level hasn’t changed?

I’d like to combine this sensor with a low pass filter to eliminate frequent “rainy/cloudy” changes and their huge jump in illuminance. However, it seems that the brightness level is only updated when a change in conditions occurred, causing my filters to not converge or update in time…

I tried setting scan_interval to 5 minutes, but that only seems to affect the minimum interval, not the maximum.

Yes, there is, but it would take a few changes. First, I’d want it to be configurable, so a new config option would have to be added. Next the should_poll method would have to change to return True if the new config option was set to True. Next an override of the force_update method would have to be added which should also return True if the new config option was set to True.

If you want to test out the theory, you could simply change the should_poll method to always return True, and add the following:

    @property
    def force_update(self):
        return True

I was interested in trying this out as well. Could you expand a little more on what should be changed? I looked at the code, and I can see this in the should_poll:

def should_poll(self):
    # For the system (i.e., EntityPlatform) to configure itself to
    # periodically call our async_update method any call to this method
    # during initializaton must return True. After that, for WU we'll
    # always poll, and for others we'll only need to poll during the ramp
    # up and down periods around sunrise and sunset, and then once more
    # when period is done to make sure ramping is completed.
    if not self._init_complete or self._using_wu:
        return True
    changing = 0 < self.sun_factor(dt_util.now()) < 1
    if changing:
        self._was_changing = True
        return True
    if self._was_changing:
        self._was_changing = False
        return True
return False

Not sure what I would change there to make it always return true? Based on my limited coding experience I would think this?

def should_poll(self):
      return True

Or should there be more to it than that? And then just add the other three lines in an @property below that?

Yes, change:

    @property
    def should_poll(self):
        # For the system (i.e., EntityPlatform) to configure itself to
        # periodically call our async_update method any call to this method
        # during initializaton must return True. After that, for WU we'll
        # always poll, and for others we'll only need to poll during the ramp
        # up and down periods around sunrise and sunset, and then once more
        # when period is done to make sure ramping is completed.
        if not self._init_complete or self._using_wu:
            return True
        changing = 0 < self.sun_factor(dt_util.now()) < 1
        if changing:
            self._was_changing = True
            return True
        if self._was_changing:
            self._was_changing = False
            return True
        return False

to:

    @property
    def should_poll(self):
        return True

    @property
    def force_update(self):
        return True

Awesome, thanks! I just went ahead and changed the script, I’ll let you know if I see anything weird or not working properly.