WARNING (MainThread) [homeassistant.config] Package homeassistant contains invalid customize

Some background:

I started splitting home assistant configuration similar to how @frenck has done in his excellent config here

I got mostly everything working fine with the exception of this weird issue that is puzzling me.

Let me start with my config/integrations/homeassistant.yaml file as I suspect that file is not properly being loaded.

---
# Sets up Home Assistant.
#
# Basically, everything that doesn't have to be in the
# main configuration.yaml.
#
# https://www.home-assistant.io/docs/configuration/basic/
#
homeassistant:
  # lets do the basic setup
  latitude: !secret zone_office_latitude
  longitude: !secret zone_office_longitude
  elevation: !secret zone_office_elevation
  time_zone: !secret zone_office_time_zone
  unit_system: metric
  name: Office

  # # Lets me customize individual entities
  #customize: !include_dir_merge_named ../customizations/entities
  #customize: !include ../customize.yaml

  # # Lets me custimize all entities in a domain
  # customize_domain: !include_dir_merge_named ../customizations/domains

  # # Lets me custimize entities based on a matching pattern
  # customize_glob: !include_dir_merge_named ../customizations/globs

  # https://www.home-assistant.io/docs/authentication/providers/
  auth_providers:
    - type: homeassistant

As you can see, I’ve commented out customizations folder inclusions to narrow down the issue.
By the way the time_zone entry is not being picked up, and the only way I can get it is by setting it in the UI under general (more on this later)

As seen in here, no customization file is referenced, however if I restart Home Assistant or reload customizations the following message is seen in the logs

WARNING (MainThread) [homeassistant.config] Package homeassistant contains invalid customize

Usually Home Assistant is very good in pinpointing the offending file / line, but not in this case.
I’ve searched all my configuration files for customize entries made sure none existed, yet I still got this error.

But it gets weirder,
In and attempt to narrow down the issue, I went to the UI and customized an entity in customization section
I got the following message:

It seems that your configuration.yaml doesn't properly include customize.yaml.
Changes made here are written in it, but will not be applied after a configuration reload unless the include is in place.

Even though it created the customize.yaml file and the settings applied (ie customization worked)
So I figured, ok it needs the inclusion under the homeassistant heading
hence why you see the commented line above
customize: !include ../customize.yaml
So I added that line (uncommented of course) and restarted Home Assistant.
Same issue, the error in the log, and also in the UI about including customize.yaml

Ok maybe it doesn’t like ../ in the path, (even though it didn’t make much sense)
I changed the entry to customize: !include customize.yaml
and restarted Home Assistant
This time I got the following error

ERROR (MainThread) [homeassistant.components.homeassistant]   in "/config/integrations/homeassistant.yaml", line 21, column 14: Unable to read file /config/integrations/customize.yaml

Which makes sense because the file did not exist, and also a proof that the homeassistant.yaml file is loading properly.

I figured, ok, let me just paste all my previous customization entries into customize.yaml, set the inclusion back to customize: !include ../customize.yaml and live with the error for now until I figure out the issue.

Needless to say I searched the community posts and found several matches that all got resolved by adding the customize.yaml inclusion.

To my surprise, none of my pasted customizations seems to have been read and applied.
The one manually done in the UI which created the initial customize.yaml file was still working.

So I checked under UI customization one of the entities that I was customizing to see if it had my setting.
To my surprise it showed the following message
image

How is that even possible? it knows about the customization, but it’s not showing up as customized on the entity (ie the entity is not showing the picture)
Ok let me save it in the UI and see what happens.
Surprise surprise, the entity image is properly displayed.
I compared the customize.yaml file before saving and after saving, 0 differences, identical, with the exception of modified date, there is zero difference.

I repeated the process, just to be sure I wasn’t imagining things, nope, if I select customization entry in the UI and hit save, it works.
Of course the logs still show the invalid customize message and the ui still says that I should include the customize.yaml (even though it is already included)

Can anyone make any sense of what’s going on? it’s a head scratcher (at least for me).

I even searched all hassio folder for any customiz reference just in case some hidden file had some setting.
Nope

It is worth to note that initially I had the homeassistant.yaml file exactly like frenck’s ie all 3 inclusions
entity, domain, glob
with my customizations in respective directories broken down by entities / domains.
I got the same errors hence why I commented the lines in an attempt to find the root issue.

I failed.

Coming back to the time_zone issue
Any clue why it’s not working?
Somehow I get a feeling that entries loaded from this homeassistant.yaml file are having issues

Hope someone has a clue.
Thanks
BB

Update:

Finally figured it out
The following lines in homeassistant.yaml

  # https://www.home-assistant.io/docs/configuration/basic/
  latitude: !secret zone_office_latitude
  longitude: !secret zone_office_longitude
  elevation: !secret zone_office_elevation
  time_zone: !secret zone_office_time_zone
  unit_system: metric
  name: Office

Are the culprit, moving them out of homeassistant.yaml and into configuration.yaml fixes both time_zone and customization issues.

Considering that according to https://www.home-assistant.io/docs/configuration/basic/
those settings belong in the homeassistant namespace same as customize as well as Authentication providers: https://www.home-assistant.io/docs/authentication/providers/
I don’t understand why it didn’t work all in homeassistant.yaml file.

The problem is back with 0.115.2, and no changes were made to customization.
I see others have the same issue, and fix almost always involves removing entries from homeassistant package and adding them to configuration.yaml under homeassistant.
Perhaps this error message is a red herring and not customize related.

Update, moving the customizations into configuration.yaml and out of homeassistant package resolved the issue.