Launched: Data Science Portal

Blog
104

Where is this priority? Hassio users get HA updates a minimum of 9 hours and sometimes even days after the actual update to HA comes out. You probably wouldn’t know this because Hassio is a black box to anyone that doesn’t use it and even to many of those that do. There are some hotfix releases Hassio doesn’t even see because by the time all the images are rendered, the next hotfix was released and available for days…

This priority is completely imagined or it’s just ignorant of how Hassio is able to exist. Home Assistant “proper” has to be there first in order for Hassio to become what is basically a curated and controlled docker environment. There is essentially one dev and there is one main source for all addons within the community repo (who has recently been joined by others and I don’t mean to leave them out because all of their effort and time is absolutely appreciated). They are extremely busy people and also have their own children and lives and work.

See this from the standpoint of a Hassio user and the pace of development will seem like it is chained to the ground for lack of resources compared to Home Assistant itself. Hassio is here because of the sheer will of a few people. The constant admonition and rebuke from those who have no idea and aren’t even “living here” looks like straight up paranoia.

A portal and an addon for hassio was announced.Well, sorry to say but Hassio has a bunch of other addons that were never announced and this apparently freaks people out. But it shouldn’t because they don’t affect Home Assistant in the slightest. Home Assistant “proper” users are still free to config their own duckdns, nginx, choose their own IDE’s, set up motion eye somewhere, or whatever the entire world of computing has to offer. Hassio users have to wait for someone to come up with it.

Home Assistant simply will not grow if it doesn’t appeal to those outside of the crowd that likes to furiously scrawl out complex templates and attach esp8266’s to anything they might stick to. Are they more basic or some kind of “lesser” user? Well, some absolutely are. There are those who expect some kind of iphone app experience and don’t know how to find an IP address on their network. Those people have a rough time no matter what installation method they choose…but many are above this level.

DrZzz, TheHookUp and even Frenck himself have seen large community growth on Youtube and Twitch in the last 6 months. I mean yeah, Frenck doesn’t have the +20,000 subscribers, but who knew that +700 people would sign up for notifications to watch someone grumble, mumble rub his eyes and code at length? :face_with_raised_eyebrow:

These viewers aren’t “basic” users watching other basic users use Hassio. These are parents and teachers and floating heads and glitter bow tie wearing grumpy geniuses who prefer to spend more time building and connecting things outside of or with Home Assistant instead of in it…and in some cases all of the above. There’s also a growing community of people who main Lovelace and and their focus seems to be making the UI look great and coming up with funky cards or design as opposed to automation itself. The community is the best community I’ve seen for just about anything. Everyone is from everywhere and it’s a wild west world of a 1000 things that don’t want to work together but do.

About less than a year ago the given number for hassio/docker combined was 1/3 of Home Assistant users by the way…I don’t know what it is now.

11 Likes
105

Certainly!

All of the necessary files can be found in my GitHub repo, so I’ll frequently point there for the details. Let’s run through what I have.

1. docker-compose.yaml

First, let’s examine an excerpt from my docker-compose.yaml, which outlines the spec for how I build a JupyterLab image:

  jupyterlab:
    build:
      context: ./jupyterlab/docker
    container_name: jupyterlab
    environment:
      JUPYTER_ENABLE_LAB: 'yes'
    image: bachya/jupyterlab
    restart: unless-stopped
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - jupyterlab-config:/home/jovyan

Three things to note here:

  1. My image comes from ./jupyterlab/docker/Dockerfile, which is based upon the minimal-notebook image from Jupyter and adds in the Python packages that the hass.io add-on utilizes.
  2. I use the JUPYTER_ENABLE_LAB environment variable to ensure the container starts in Lab mode.
  3. I define a volume (jupyterlab-config) and map it to /home/jovyan, which is where JupyterLab stores its data.

2. jupyter_notebook_config.py

I use the same settings that the hass.io add-on uses.

3. Home Assistant Panel

I use NGINX as a reverse proxy to my various Docker-based services, including JupyterLab. My NGINX configuration for this service looks like this:

server {
  listen 443 ssl;
  server_name 98asdasdfyu2.myserver.com;

  include /etc/nginx/security.conf;
  include /etc/nginx/ssl.conf;

  location / {
    proxy_pass http://jupyterlab:8888;
  }
}

Notice that my custom URL – 98asdasdfyu2.myserver.com – proxies traffic to the jupyterlab container on port 8888.

I then use this URL in a Panel iFrame:

jupyterlab:
  title: JupyterLab
  url: !secret panel_url_jupyterlab  # URL GOES HERE
  icon: mdi:database-search

That’s about it! Let me know what questions you have.

13 Likes
106

Thankyou. hopefully this will help someone out there with using docker

2 Likes
107

I think it comes down to how people have approached home automation tbh. I, like many others in this thread, was here way before hassio even became a thing. Then when it did become a thing it was only available as an image for raspbery pi, but most of us had outgrown the pi. Then when hassio became something that could be installed elsewhere, I couldn’t try it out because I use bluetooth for presence detection and hassio could only do this if you were using a rasperry pi’s internal bluetooth module. Then by the time that problem was sorted, there was another thing that stopeed me moving, and another, and another…

I think about making the switch maybe once a month, and every time I decide against it. There was a massive security flaw in the SAMBA addon not so long back and people were getting hacked left, right and centre. I didn’t get hacked because I set up my own system and it was secure. Now upon realising this flaw was present the devs fixed it, but what if the maintainer for that addon couldn’t be bothered. You would have to discontinue use of the addon and hope somebody else came along to make you a new one. Or you would have to use a docker container that you configure yourself, and if you;re going to do that, you’re basically just using homeassistant without the ‘benefits’ of hassio.

There are lots of people coming to the forum/reddit regularly completely locked out of their hassio systems because the SSH addon isn’t working, or they haven’t set it up properly, or XYZ. I don’t get locked out of my system, because I built it myself and I can SSH in to the device itself independently of homeassistant. You can do this too if you’re running Hassio in a VM, but then, as above, you’re basically using homeassistant without utilising the ‘benefits’ of hassio.

Based on the above two paragaphs alone I feel I’m better off sticking with ‘normal’ homeassistant.

In addition to this, from a personal standpoint I think the amount of effort that has gone in to making ‘hassio’ , then making hassio ‘user friendly’ could have been better spent making ‘homeassistant’ user friendly. For example the automation editor was introduced in 0.45 - there have been 205 releases since then (according to PyPi), and still the OR condition isn’t supported. There’s nearly 800 live bug reports that barely seem to get any attention.

It’s lovely getting all of these ‘new features’ all the time, but I worry that we’re so obsessed with new features we’ve forgotten that a load of the old ones don’t work properly. I would have thought that the automation editor should have been the priority for getting new users on board, rather than a ‘data portal’ that can tell them how they could automate it better if only they could use an OR condition.

From the moment hassio was introduced there was an extra level of complexity added. When there was just homeassistant, there were several install methods but everything was the same. Now it’s hard to work out what’s going on for each user that comes here asking for support and I just don’t think the benefits have outweighed the downsides. I think if all of that effort had been put in to making homeassistant easier to use, you and your mates could still have used it just as easily as you do now with hassio, but there would be a lot less angst in the community, and it would e a lot easier to get help when something went wrong.

I’m rambling now, time to finish up…

I love this platform, I dedicate a lot of time to helping others to use this platform, update documentation to make it easier for people to use this platform, even occasionally try to go right out of my comfort zone and make some minor code changes to improve this platform. I even bought the tee-shirt (see profile pic!!). I don’t want to see it go up the spout because it is so fragmented, but if I was a betting man I’d put money on that being what kills it eventually.

9 Likes
108

This is a misunderstanding. SSH to the host has a “hassio debug” designation and a scary warning but can easily be configured. What tricks people up is that they expect a full blown OS but it is minimal…mainly for dbus and docker. Also…it’s docker. So even with a full OS as in a VM, everything is images and containers and not traditional. Fun stuff to learn about, though. And it’s been interesting to watch the progress. Things aren’t getting more restrictive but less over time. I’m not a docker expert but Hassio has exposed me to it in bits and pieces. I now know how to list containers, see docker stats, but have never needed to use compose or figure out docker internal networking, for example.

109

Whoever knew having unsecured Samba shares with no password open to the Internet was a bad idea right?

Don’t know about people getting hacked left and right… there may have been 2 that I remember and one who ended up admitting that his father had an echo spot that still had his credentials in it so his lights were turning on and off… It was not a big issue as I recall but did result in security being forced on people… and that is always so welcome by some as well… I still hear the bitching about the new auth system being ‘forced’ on local users.

There’s ways to do a generic linux hassio install so you get the best of hassio and it’s addons as well as a full linux system. Best of all worlds.

People tend to look down their noses at Hassio but I find it suits me perfectly. There are ways to do anything you can do with addons in straight linux anyway so if you like tinkering you can get the same result but hassio makes home assistant much more accessible to the masses in my opinion.

3 Likes
110

I don’t believe it will. Look at the likes of Kodi, when they started developing for specific platforms people were the same saying it will be the end of it. its still going and getting stronger.
My biggest concern, is that the main developers get so hacked off when they try to implement something and it triggers negative responses that they leave the project to die. I give a classic example of another open source project MediaPortal, which for those that don’t know was a fork by one of the original developers of the original XBMC the forerunner to Kodi. After lots of negativity on the forums on where he was trying to go with the project he up and left. Since then i believe it never recovered and is slowly dying a slow death.

Lets not let this happen here, we should be supportive on where they want to take it, it is after all free and done with their free time

3 Likes
111

Well said on all points. I think my journey has been a little bit like yours, but led me down a different path. I initially installed hassio on a Pi zero, just to see wtf hassio was. However, I ran out of horsepower almost immediately, ditched that, changed to a Pi 2. Ran out again, changed to Pi 3. Ran for for a while, but was always running into things I couldn’t do because I didn’t have all of the access running it that way allows. And now, for probably almost a year, I’ve been running it on Ubuntu. In my case, it is the best scenario for me, because I can use hassio (and the hassio CLI management interface) AND, I can access the OS directly via CLI on a different port. So, if my hassio dies, for some reason, I still have access to the box, and I can whack away at the docker containers and files until I can figure out what killed it.

I hope you are wrong about this dying one day…I don’t see anything on the market free OR paid, which comes close to what home assistant can do. Anyway, now I’M rambling.

112

Oh, I hope I’m wrong on my final point too btw :slight_smile:

113

@bachya can you be persuaded to make a PR to the docs with your docker instructions?

OK it would be nice to talk about some data science on this thread now :slight_smile:
I am particularly interested in monitoring and predicting energy usage, with an interesting study here. Is anyone monitoring their energy usage and have a decent dataset?

4 Likes
114

Talking about data science would be great. Maybe we should start a fresh thread and let this topic here dry out.

3 Likes
115

For those who want to seperate datascience and home automation on seperate systems, i would recommend to install it on your computer for testing. Here simple steps to set it up. (5 min)
This assumes you already have python installed on your machine. This procedure could also be used with small modifications on your existing home assistant virtual env on the pi or other OS.

Data environment on Windows

1 Create a directory for virtual data environment

Start Command prompt (in administrator mode):

Create directory for virtual environment:

mkdir c:\hassdata

2 Create virtual pyton environment

$ virtualenv c:\hassdata

3 Activate virtual environment

$ c:\hassdata\scripts\activate

Will show:

(hassdata)$

4 Install jupyterlabs

(hassdata)$pip install jupyterlab

5 Install hass datascripts

(hassdata) $pip install HASS-data-detective

6 Run jupiterlabs

(hassdata) $ jupyter lab

3 Likes
116

Please do not panic. Hassio and home assistant are doing fine as it is.
The people who wish a full blown Linux system just do not know docker enough.

The industry in that space is also transforming in that direction. Look at rancher 2 with rancher OS and core OS for example.
These are minimal images where almost every process is a docker container to run kubernetes workloads on really small systems.
It is not that easy in the beginning because you have to learn something new.
But it has way less components and so less potential failures.
That is Hass OS.

Hassio is just a provisioner for home assistant docker containers which runs on Hass OS and any Linux system as well.
It is like kubernetes but less complex which in my opinion is good because kubernetes is a hell of a steep learning curve especially for beginners.

Almost everything you can do with home assistant on a Linux system you can do with Hassio on Hass OS or Hassio on on a Linux system.
It is just different and maybe a little more complex to accomplish it with Hassio since docker is another layer of complexity.

3 Likes
117

Hi @bachya I was looking for your image on dockerhub but could not find it.
Is it somewhere else?

EDIT: I see, you build the docker yourself on startup?
Would it be possible to provide it as an image in dockerhub?
I use unRAID and would love to create a template for it so we can all use it, if that is OK with you?

118

The compose file builds it based on the dockerfile in ./jupyterlab/docker

1 Like
119

Thanks!
Just worked that out as well and updated my post.
I use unRAID, need to see if I can do that there …

120

I’ve been tracking most of the energy consumption in my home for the past 3 months or so. Also have outdoor and room-by-room temperatures to parallel that data, so this sounds really interesting to me. Looks like tomorrow morning will be a perfect time to spin up JupyterLab in Docker and see what kind of interesting patterns might be useful to future automations!

2 Likes
122

Where do you configure or set these settings?

123

`I’m an happy Hassio user and I’m excited by the idea to do some deep analytics (not only monitoring by Grafana) on my HA data. Unfortunately, from yesterday, I’ trying to install the add-on but I’m continuing to take the following error:

ERROR (SyncWorker_8) [hassio.docker.interface] Can’t install hassioaddons/jupyterlablite:0.1.0 -> 404 Client Error: Not Found (“no such image: hassioaddons/jupyterlablite:0.1.0: No such image: hassioaddons/jupyterlablite:0.1.0”).

I’m using Pine64 with Dietpi OS. HA 0.84.6 and Hassio supervisor is 141.

I really appreciate your help to fix the problem. Thank you very much in advance

ADDED INFO:
I tried to pull manually the image with the command:

  • docker pull hassioaddons/jupyterlablite:0.1.0

and I obtain the following message:

  • no matching manifest for unknown in the manifest list entries`

I kindly ask you a little help to fix my problem.
Thank you in advance

124

I threw together a Dockerfile that is auto building to https://hub.docker.com/r/kylerw/hass-data-detective for those that want to take advantage of it.

I have this running on unRAID in Bridge mode and map

Volumes

/etc/localtime:/etc/localtime:ro
/mnt/cache/appdata/jupyter:/home/jovyan 
/mnt/cache/appdata/home-assistant:/hass-config:ro

Port
8888:8888

and

Variable

JUPYTER_ENABLE_LAB:yes

screenshot of the Docker setup:

image
image992×646 34.9 KB

then in the Jupyter Notebooks I use:

from detective.core import db_from_hass_config
db = db_from_hass_config("/hass-config")

to detect the database I’m using (separate mariadb container)

On first run, you need to view the logs to get the token that is generated to log in and create a password.

4 Likes