Dashboard 'robots.txt' not found

not the wisest move.
that way you override stuff that can cause trouble.

read the first 2 lines :wink:
but oke, please change that dirname to a skinname (like my_default)
and there also shouldnt be all kind of css files in the custom_css
they can all lead to trouble.

did you add those on purpose? and whats that purpose?
if you dont need them, delete them.

I just wanted to change the background color to black…

Is there another way to do that besides creating a new skin?

the problem is that you created a custom skin, and now you cant use the default anymore.
it is possible the way you do it, but you need to remember that you need to update and change the files, every time you update appdaemon, to make sure you dont have a problem because you are using an old default skin.

to make sure that your troubles dony come from something you created yourselve somehow, you always need to go back to the basic settings that the programmer created.
thats only possible if you rename the skin.

Ok, that won’t be a problem.

The only thing that is bugging me now is the warning about robots.txt

Thank you very much for you patience and time!

did you check with filezilla in the dashboards dir?
the file or a refference to it must be somewhere in your dashboards dir, or in custom css or in custom widgets

it bugs me too, that something like that exists and cant be found.

I’ve searched the file inside all folder and subfolders of appdaemon. If i knew the content of the file i could create it and try to put on the right place… What’s strange is that everything works great, no problem besides the warning. And the pages are a little slow to load.

the content isnt important and there is no right place.
its a file that should not exist.

But is it a problem created by anything i’ve done?

thats the problem, i cant find that out unless you delete everything that you added yourself.
you could backup the files from inside custom css and custom widgets and then delete everything in those dirs.
clear the dashboard dir except for 1 simple dashboard
clear the apps dir

then restart appdaemon,
if the error then dissapears it is something you created.
if not then i can only assume that there is something gone wrong during install from the addon and you could try to uninstall and reinstall.

Ok, i’ve taken this steps:

  • Stopped AD3
  • Backup the appdaemon folder
  • Removed the folders apps, custom_css and custom_widgets. Now i only have the dashboard and logs folders
  • Deleted the content of the dashboard folder. Created a simple dashboard file with only the clock widget
  • Deleted the content of the log files.
  • Uninstalled AD3 and reinstalled again (frenck)
  • Started AD3

appdaemon.log:

2018-03-08 11:20:13.091755 INFO AppDaemon Version 3.0.0b5 starting
2018-03-08 11:20:13.094045 INFO Configuration read from: /config/appdaemon/appdaemon.yaml
2018-03-08 11:20:13.099253 INFO AppDaemon: Starting Apps
2018-03-08 11:20:13.114559 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
2018-03-08 11:20:13.232376 INFO AppDaemon: HASS: HASS Plugin Initializing
2018-03-08 11:20:13.234591 INFO AppDaemon: HASS: HASS Plugin initialization complete
2018-03-08 11:20:13.236701 INFO Starting Dashboards
2018-03-08 11:20:13.272007 INFO API is disabled
2018-03-08 11:20:13.310878 INFO AppDaemon: HASS: Connected to Home Assistant 0.64.3
2018-03-08 11:20:14.802660 INFO AppDaemon: Got initial state from namespace default
2018-03-08 11:20:16.102113 INFO AppDaemon: Adding /config/appdaemon/apps to module import path
2018-03-08 11:20:16.116628 INFO AppDaemon: App initialization complete
  • Open the dashboard

access.log

2018-03-08 11:22:47.861592 INFO Compiling dashboard 'robots.txt'
2018-03-08 11:22:49.677347 WARNING Dashboard 'robots.txt' not found
2018-03-08 11:22:49.679494 WARNING ------------------------------------------------------------
2018-03-08 11:22:49.680634 WARNING Unexpected error during DASH creation
2018-03-08 11:22:49.681711 WARNING ------------------------------------------------------------
2018-03-08 11:22:49.693481 WARNING Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/appdaemon/dashboard.py", line 822, in get_dashboard
    with open(include_path, "r") as include_file:
FileNotFoundError: [Errno 2] No such file or directory: '/config/appdaemon/compiled/html/default/robots.txt_head.html'

2018-03-08 11:22:49.694622 WARNING ------------------------------------------------------------
2018-03-08 11:22:49.695576 INFO function [get_dashboard] finished in 2204 ms
2018-03-08 11:22:49.790877 INFO Compiling dashboard 'robots.txt'
2018-03-08 11:22:51.042026 WARNING Dashboard 'robots.txt' not found
2018-03-08 11:22:51.046203 WARNING ------------------------------------------------------------
2018-03-08 11:22:51.048035 WARNING Unexpected error during DASH creation
2018-03-08 11:22:51.050019 WARNING ------------------------------------------------------------
2018-03-08 11:22:51.052977 WARNING Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/appdaemon/dashboard.py", line 822, in get_dashboard
    with open(include_path, "r") as include_file:
FileNotFoundError: [Errno 2] No such file or directory: '/config/appdaemon/compiled/html/default/robots.txt_head.html'

2018-03-08 11:22:51.054758 WARNING ------------------------------------------------------------
2018-03-08 11:22:51.056782 INFO function [get_dashboard] finished in 1296 ms
2018-03-08 11:22:51.092841 INFO Compiling dashboard 'test'
2018-03-08 11:22:51.153626 INFO Compiling dashboard 'robots.txt'
2018-03-08 11:22:53.748738 WARNING Dashboard 'robots.txt' not found
2018-03-08 11:22:53.782265 WARNING ------------------------------------------------------------
2018-03-08 11:22:53.788971 WARNING Unexpected error during DASH creation
2018-03-08 11:22:53.796282 WARNING ------------------------------------------------------------
2018-03-08 11:22:53.805927 WARNING Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/appdaemon/dashboard.py", line 822, in get_dashboard
    with open(include_path, "r") as include_file:
FileNotFoundError: [Errno 2] No such file or directory: '/config/appdaemon/compiled/html/default/robots.txt_head.html'

2018-03-08 11:22:53.813977 WARNING ------------------------------------------------------------
2018-03-08 11:22:53.820703 INFO function [get_dashboard] finished in 2725 ms
2018-03-08 11:22:54.786233 INFO function [get_dashboard] finished in 4147 ms
2018-03-08 11:22:55.125523 INFO Compiling dashboard 'robots.txt'
2018-03-08 11:22:55.537756 INFO HADashboard: New dashboard connected: Test Panel
2018-03-08 11:22:56.413350 WARNING Dashboard 'robots.txt' not found
2018-03-08 11:22:56.415621 WARNING ------------------------------------------------------------
2018-03-08 11:22:56.417094 WARNING Unexpected error during DASH creation
2018-03-08 11:22:56.419249 WARNING ------------------------------------------------------------
2018-03-08 11:22:56.422244 WARNING Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/appdaemon/dashboard.py", line 822, in get_dashboard
    with open(include_path, "r") as include_file:
FileNotFoundError: [Errno 2] No such file or directory: '/config/appdaemon/compiled/html/default/robots.txt_head.html'

2018-03-08 11:22:56.423719 WARNING ------------------------------------------------------------
2018-03-08 11:22:56.425158 INFO function [get_dashboard] finished in 1388 ms

AD3 addon log

[cont-init.d] 00-banner.sh: executing... 
-----------------------------------------------------------
 Hass.io Add-on: AppDaemon3 v0.3.0
 Python Apps and HADashboard using AppDaemon 3.x for Home Assistant
 From: Community Hass.io Add-ons
 By: Franck Nijhof <[email protected]>
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing... 
Log level is set to INFO
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] 02-updates.sh: executing... 
INFO: You are running the latest version of this add-on
[cont-init.d] 02-updates.sh: exited 0.
[cont-init.d] 03-version-requirements.sh: executing... 
INFO: Supervisor version requirements checks passed.
[cont-init.d] 03-version-requirements.sh: exited 0.
[cont-init.d] 20-init-configuration.sh: executing... 
[cont-init.d] 20-init-configuration.sh: exited 0.
[cont-init.d] 21-compiled-dir.sh: executing... 
[cont-init.d] 21-compiled-dir.sh: exited 0.
[cont-init.d] 30-auto-password.sh: executing... 
[cont-init.d] 30-auto-password.sh: exited 0.
[cont-init.d] 31-ha-url.sh: executing... 
[cont-init.d] 31-ha-url.sh: exited 0.
[cont-init.d] 50-compiled-symlink.sh: executing... 
[cont-init.d] 50-compiled-symlink.sh: exited 0.
[cont-init.d] 80-system-packages.sh: executing... 
[cont-init.d] 80-system-packages.sh: exited 0.
[cont-init.d] 81-python-packages.sh: executing... 
[cont-init.d] 81-python-packages.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
starting version 3.2.4
[services.d] done.
Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 50, in wrapper
    return await myfunc(*args)
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 253, in load_dash
    return web.Response(text=response, content_type="text/html")
  File "/usr/lib/python3.6/site-packages/aiohttp/web_response.py", line 471, in __init__
    type(text))
TypeError: text argument must be str (<class 'dict'>)
Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 50, in wrapper
    return await myfunc(*args)
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 253, in load_dash
    return web.Response(text=response, content_type="text/html")
  File "/usr/lib/python3.6/site-packages/aiohttp/web_response.py", line 471, in __init__
    type(text))
TypeError: text argument must be str (<class 'dict'>)
Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 50, in wrapper
    return await myfunc(*args)
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 253, in load_dash
    return web.Response(text=response, content_type="text/html")
  File "/usr/lib/python3.6/site-packages/aiohttp/web_response.py", line 471, in __init__
    type(text))
TypeError: text argument must be str (<class 'dict'>)
Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 50, in wrapper
    return await myfunc(*args)
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 253, in load_dash
    return web.Response(text=response, content_type="text/html")
  File "/usr/lib/python3.6/site-packages/aiohttp/web_response.py", line 471, in __init__
    type(text))
TypeError: text argument must be str (<class 'dict'>)

So… uninstalled and reinstalled, didn’t use any custom stuff, used a simple dashboard and there is always that warning about robots.txt

only thing i can think of is that there is still a hidden file in the config dir somewhere and you didnt delete it completely.

you could delete the complete config dir and all its subdirs.
and the reinstall.
the config dir should be recreated.

because i havent seen that error from anyone else, a complete clean reinstall cant have that error.

but that is about the last possible thing i can think of.

That’s a good idea. What I’ve done was:

  • Uninstalled AD3
  • Deleted the appdaemon folder completely
  • Reinstalled AD3 (frenck’s)
  • Started AD3

On the log i could see AD complaining about the the folder not beeing there and created it. I think this is the line:

INFO: ha_key is missing in the AppDaemon configuration, fixing...

And then this:

2018-03-08 12:02:43.290924 INFO AppDaemon Version 3.0.0b5 starting
2018-03-08 12:02:43.291995 INFO Configuration read from: /config/appdaemon/appdaemon.yaml
2018-03-08 12:02:43.296337 INFO AppDaemon: Starting Apps
2018-03-08 12:02:43.309498 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
2018-03-08 12:02:43.436144 INFO AppDaemon: HASS: HASS Plugin Initializing
2018-03-08 12:02:43.438028 INFO AppDaemon: HASS: HASS Plugin initialization complete
2018-03-08 12:02:43.440423 INFO Starting Dashboards
2018-03-08 12:02:43.479128 INFO API is disabled
2018-03-08 12:02:43.521039 INFO AppDaemon: HASS: Connected to Home Assistant 0.64.3
2018-03-08 12:02:45.603249 INFO AppDaemon: Got initial state from namespace default
2018-03-08 12:02:46.155206 INFO AppDaemon: Reading config
2018-03-08 12:02:46.170210 INFO AppDaemon: /config/appdaemon/apps/apps.yaml added or modified
2018-03-08 12:02:46.172676 INFO AppDaemon: /config/appdaemon/apps/apps.yaml added or modified
2018-03-08 12:02:46.173705 INFO AppDaemon: App 'hello_world' added
2018-03-08 12:02:46.175648 INFO AppDaemon: Adding /config/appdaemon/apps to module import path
2018-03-08 12:02:46.178251 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/hello.py
2018-03-08 12:02:46.254758 INFO AppDaemon: Initializing app hello_world using class HelloWorld from module hello
2018-03-08 12:02:46.949837 INFO hello_world: Hello from AppDaemon
2018-03-08 12:02:46.961247 INFO hello_world: You are now ready to run Apps!
2018-03-08 12:02:46.963341 INFO AppDaemon: App initialization complete
2018-03-08 12:02:52.986258 INFO HADashboard: New dashboard connected: Test Panel

Then, opened the test dashboard and could see the hello world dash.

But the AD addon log shows:

2018-03-08 12:06:24.851444 INFO function [get_dashboard] finished in 713 ms
Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 50, in wrapper
    return await myfunc(*args)
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 253, in load_dash
    return web.Response(text=response, content_type="text/html")
  File "/usr/lib/python3.6/site-packages/aiohttp/web_response.py", line 471, in __init__
    type(text))
TypeError: text argument must be str (<class 'dict'>)
2018-03-08 12:06:26.180715 INFO Compiling dashboard 'robots.txt'
2018-03-08 12:06:27.832370 WARNING Dashboard 'robots.txt' not found
2018-03-08 12:06:27.870765 WARNING ------------------------------------------------------------
2018-03-08 12:06:27.885316 WARNING Unexpected error during DASH creation
2018-03-08 12:06:27.891740 WARNING ------------------------------------------------------------
2018-03-08 12:06:27.899650 WARNING Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/appdaemon/dashboard.py", line 822, in get_dashboard
    with open(include_path, "r") as include_file:
FileNotFoundError: [Errno 2] No such file or directory: '/config/appdaemon/compiled/html/default/robots.txt_head.html'
2018-03-08 12:06:27.905454 WARNING ------------------------------------------------------------
2018-03-08 12:06:27.907186 INFO function [get_dashboard] finished in 2000 ms
Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 50, in wrapper
    return await myfunc(*args)
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 253, in load_dash
    return web.Response(text=response, content_type="text/html")
  File "/usr/lib/python3.6/site-packages/aiohttp/web_response.py", line 471, in __init__
    type(text))
TypeError: text argument must be str (<class 'dict'>)
2018-03-08 12:06:27.959026 INFO Compiling dashboard 'robots.txt'
2018-03-08 12:06:28.575948 WARNING Dashboard 'robots.txt' not found
2018-03-08 12:06:28.577614 WARNING ------------------------------------------------------------
2018-03-08 12:06:28.578374 WARNING Unexpected error during DASH creation
2018-03-08 12:06:28.579001 WARNING ------------------------------------------------------------
2018-03-08 12:06:28.580207 WARNING Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/appdaemon/dashboard.py", line 822, in get_dashboard
    with open(include_path, "r") as include_file:
FileNotFoundError: [Errno 2] No such file or directory: '/config/appdaemon/compiled/html/default/robots.txt_head.html'
2018-03-08 12:06:28.580851 WARNING ------------------------------------------------------------
2018-03-08 12:06:28.581572 INFO function [get_dashboard] finished in 650 ms
Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 50, in wrapper
    return await myfunc(*args)
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 253, in load_dash
    return web.Response(text=response, content_type="text/html")
  File "/usr/lib/python3.6/site-packages/aiohttp/web_response.py", line 471, in __init__
    type(text))
TypeError: text argument must be str (<class 'dict'>)
2018-03-08 12:06:28.604239 INFO Compiling dashboard 'Hello'
2018-03-08 12:06:29.459935 INFO Compiling dashboard 'robots.txt'
2018-03-08 12:06:31.612488 WARNING Dashboard 'robots.txt' not found
2018-03-08 12:06:31.645461 WARNING ------------------------------------------------------------
2018-03-08 12:06:31.652063 WARNING Unexpected error during DASH creation
2018-03-08 12:06:31.658192 WARNING ------------------------------------------------------------
2018-03-08 12:06:31.670327 WARNING Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/appdaemon/dashboard.py", line 822, in get_dashboard
    with open(include_path, "r") as include_file:
FileNotFoundError: [Errno 2] No such file or directory: '/config/appdaemon/compiled/html/default/robots.txt_head.html'
2018-03-08 12:06:31.678046 WARNING ------------------------------------------------------------
2018-03-08 12:06:31.679045 INFO function [get_dashboard] finished in 3082 ms
Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 50, in wrapper
    return await myfunc(*args)
  File "/usr/lib/python3.6/site-packages/appdaemon/rundash.py", line 253, in load_dash
    return web.Response(text=response, content_type="text/html")
  File "/usr/lib/python3.6/site-packages/aiohttp/web_response.py", line 471, in __init__
    type(text))
TypeError: text argument must be str (<class 'dict'>)
2018-03-08 12:06:32.159121 INFO function [get_dashboard] finished in 3726 ms
2018-03-08 12:06:32.620295 INFO HADashboard: New dashboard connected: Hello Panel

Is it safe to say that there is nothing wrong on my side but a problem with the addon or AD itself?

its not an AD file, from that i am 100% sure
and you had the problem with AD 2 and AD 3, so it also cant be the addon.
that besides the fact that you are the only 1 that got those errors.

at the moment i am even wondering if your device couldnt be infected with something?
or maybe with the uninstall cashed files are not all deleted?

but whatever it is i am out of ideas

And how can i check that?

like i said i am out of ideas.
all i can think of is formatting the SD and create a clean new install and then add your config again.
but if its an infection, then even copying config files would be a risk.

i must say i never heard from an infection, and i dont even know if thats possible.
but at this moment i am just grasping.

Could this problem came from an addon or custom component?

you dont have a custom component now in the addon, so we can rule that out.
(again, unless temp files are not deleted with uninstall)

could it come from another addon?
i have no clue.
i guess it shouldnt, because addons should be seperate from each other, when i have understand it correct.
but its hassio, i guess anything is possible.

Where are those temp files located? Can they be accessed from the command line with Putty for example?

I use a few addons but they are very common (AppDaemon3, DuckDNS, Grafana, HASS Configurator, InfluxDB, Mosquitto, SSH Server and Samba Share). the only less common addon is Motion…

i have no clue where they relocated the temp files.
with normal install from appdaemon they are in the config/compiled dir.
but we noticed that they are not there.
and i dont know what else they relocated

and i am sorry but i have no clue how they did built the whole structure, and how to access the complete file structure.
hassio is like windows, you install programs, but if something is wrong, you dont want to try to find the failure :wink:
it has advantages for people who want things simple, but people who want to do more complicated stuff dont use it. (and that also creates a problem, because Andrew doesnt use it, I dont use it, so we wont get exp. in it.)