Classifying the Internet of Things

Blog
1

The core of home automation is knowing what’s going on. The faster we know about a state change, the better we can serve the user. If you want to have your lights to turn on when you arrive at home, it doesn’t help if it only knows about it after you’ve already opened the door and manually (!!) turned on the light.

Each smart device consists of the ‘normal’ device and the piece that makes it ‘smart’: the connectivity. The connectivity part of a device can consists of either control, state or both.

State describes what a device is up to right now. For example, a light can be on with a red color and a medium brightness.

Control is about controlling the smart device by sending commands via an API. These commands can vary from configuring how a device works till mimicking how a user would interact with a device. A media player can allow skipping to the next track and a sensor could allow to configure its sensitivity or polling interval.

The Home Assistant APIs are setup to be as convenient as possible. However, a network is always as weak as it’s weakest link. In our case these are the integrations. Take for example controlling a light that does not report state. The only state Home Assistant can report on after sending a command is the assumed state: what do we expect the state of the light to be if the command worked.

We want our users to get the best home automation experience out there and this starts with making sure they have devices that work well with Home Assistant. That’s why we will start applying the following classifiers to our integrations:

Classifier Description Assumed State We are unable to get the state of the device. Best we can do is to assume the state based on our last command. Cloud Polling Integration of this device happens via the cloud and requires an active internet connection. Polling the state means that an update might be noticed later. Cloud Push Integration of this device happens via the cloud and requires an active internet connection. Home Assistant will be notified as soon as a new state is available. Local Polling Offers direct communication with device. Polling the state means that an update might be noticed later. Local Push Offers direct communication with device. Home Assistant will be notified as soon as a new state is available.

The background to how we got to these classifiers can be read after the break.

State

How state is communicated can be broken down into 5 categories. They are not mutually exclusive - a device state can be available both via the cloud and local connectivity.

No state available

These are devices that do not have the capabilities to make their state available. They only allow to be controlled. For example, devices with infrared remote controls like TVs and ACs. You can press the turn on button on the remote but can only assume that your command was received and executed successfully. The device might not be powered or something is blocking the infrared receiver.

Home automation will have to approach such devices based on the assumption that it’s commands are received correctly: using optimistic updates. This means that after sending a command it will update the state of the device as if the command was received successfully.

Advantages:

Disadvantages:

  • Home automation will assume the wrong state if the command is not received correctly or if the device is controlled in any other way outside of the home automation system.

Polling the cloud

These are devices that will only report their state to their own cloud backend. The cloud backend will allow reading the state but will not notify when a new state has arrived. This requires the home automation to check frequently if the state has been updated.

Advantages:

  • Able to control devices while at home or away.
  • Cloud has access to more computing power to mine the device data to suggest optimizations to the user.

Disadvantages:

  • It doesn’t work if the internet is down or the company stops support.
  • You are no longer in control about who has access to your data.

Cloud pushing new state

All off the previous section applies to this one. On top of that the cloud will now notify the home automation when a new state has arrived. This means that as soon as the cloud knows, the home automation knows.

Advantages:

  • New state known as soon as available in the cloud.

Polling the local device

These devices will offer an API that is locally accessible. The home automation will have to frequently check if the state has been updated.

Advantages:

  • Does not depend on the internet

Disadvantages:

  • To be pollable, a device needs to be always online which requires the device to be connected to a power source.

Local device pushing new state

The best of the best. These devices will send out a notice when they get to a new state. These devices usually use a home automation protocol to pass it’s message to a hub that will do the heavy lifting of managing and notifying subscribers

Advantages:

  • Near instant delivery of new states.
  • Able to get a long battery life by going into deep sleep between state updates.

Disadvantages:

  • If it does not also support polling, home automation will not be made aware of the state after booting up until it changes.
  • If using deep sleep and wifi, will suffer a delay when waking up because connecting to WiFi and receiving an IP takes time.

Control

Controlling a device can, just like state, be done through cloud and/or local connectivity. But the more important part of control is knowing if your command was a success and the new state of the device.

No control available

These devices are not able to be controlled. They will only offer state.

Poll State after sending command

These devices will require the state to be polled after sending a command to see if a command was successful.

Advantages:

  • The state will be known right after the command was issued.

Disadvantages:

  • It can take time before the state gets updated. How often do we poll and how long do we wait till we consider the command failed? Also, a state may change because of other factors. Difficult to determine if the updated state is because of our command.

Device pushes state update

These devices will not return a new state as a result of the command but instead will push a new state right away. The downside of this approach is that we have to assume that a state update coming in within a certain period of time after a command is related to the command.

Command returns new state

The very best. These devices will answer the command with the new state after executing the command.

Classifying Home Assistant

Home Assistant tries to offer the best experience possible via its APIs. There are different ways of interacting with Home Assistant but all are local.

  • State polling is available via the REST API
  • There is a stream API that will push new states as soon as they arrive to subscribers. This is how the frontend is able to always stay in sync.
  • Calling a service on Home Assistant will return all states that changed while the service was executing. This sadly does not always include the new state of devices that push their new state, as they might arrive after the service has finished.
This is a companion discussion topic for the original entry at https://home-assistant.io/blog/2016/02/12/classifying-the-internet-of-things/
2

Is “local” always local? For example, what if your “local polling” or “local push” device requires that you use the manufacturer’s smartphone app to join it to your wireless network or for other aspects of initial setup or ongoing tuning/configuration? Worse yet, what if that app requires you to log in to the manufacturer’s web service or simply doesn’t doesn’t work without a connection to their servers? What do you do when they shut off the web service upon which that app depends? Many Matter and Homekit devices seem to fall into this category.

I think it would be helpful if the “local” classifications were slightly more specific.

1 Like
3

canvas (2)
canvas (2)2048×529 110 KB

The original post is seven years old, for goodness sake!

Start a new thread. :laughing:

5 Likes
4

Why a new thread? Why generate more when the context of the above makes clear the question and provides good doc. Naively, I think this is the right thread to have the discussion on. The details in the OP haven’t changed to my knowledge so none of the context is outdated.

2 Likes
5

So, all of the integration information pages have a link like the one circled here:

image
image925×320 62.9 KB

Which takes you to the following documentation, which you might not immediately realize is in the form of a blog post, due to use of an anchor link…

image
image612×543 63.6 KB

Further down that page, you’ll see an invitation to discuss the documentation…
image

… which brought me here, and it was only then that I noticed the text about this being a “companion discussion topic” for a blog post, but since it seemed like the blog was being used as a CMS, with the post in question serving as the official documentation on the topic, I figured it was appropriate to comment here.

If the documentation ever gets updated, perhaps it will be done with a new blog post, in which case any further discussion will be tied to a new thread.

I appreciate the importance of politely educating newcomers about posting etiquette, and I understand the reasons why it is often undesirable to have people re-opening old threads, and, but I’m not sure those reasons apply in this case.

In any event, what’s done is done, and I’d love to get some feedback on what I posted and not have this turn into a discussion of proper forum use. (I might suggest that anyone interested in that topic create a new thread!)

5 Likes
6

Absolutely right. My apologies. :flushed:

3 Likes
7

This is a very good point, and deserves more clarity on the integration pages.

One device I have that fits this description is a Yeelight desk lamp. The Yeelight integration describes itself as “Local Push”, which is true, once you manage to complete their initial setup, which was an absolute bear. It took at least 20 attempts of resetting everything with the light, the app, and my account, before I could successfully connect and enable “LAN Control”, which is required for the HA integration to work. The integration docs do mention that setup is required through the app, at least.

After blissfully working for a couple months, the light factory reset itself, and I have been dreading having to go through that setup again. It’s almost not worth the trouble of having a smart light.

Another risk is the manufacturer pushing a firmware update through the app at some point in the future which disables LAN control before I can enable it. The required app setup is a big downside to using devices like this. Flashing the internal microcontroller with ESPHome or Tasmota is a permanent solution, but not always possible, and certainly not as straightforward.

2 Likes
8

Good point is that this information do not belong only in blog post from 7-years ago but instead should really be part of the official Home Assistant documentation (which it is not) → Documentation - Home Assistant or at the very least in the Home Assistant Developer Docs (which now only refer to this blog post) → https://developers.home-assistant.io/

I by the way asked a related question about the IoT classification of the ZHA integration 2-years ago and got no reply as of yet, see:

1 Like
9

I’m not sure how you even have that question. It’s both local polling and local push. If you read the blurbs at face value, you can’t state that it’s local push when some things on it may require polling. Logically you’d have to list local polling and the website only supports 1 at a time.

10

If so then why not add both to the website? :face_with_raised_eyebrow: That is, why not make the website support more than one at a time?

2 Likes
11

Does it really matter that much? I’m sure you could make a PR to make that happen, but I’m not really seeing this as a high priority.

There’s a new doc dev, I’m sure you could easily make the case.

12

After blissfully working for a couple months, the light factory reset itself, and I have been dreading having to go through that setup again. It’s almost not worth the trouble of having a smart light.

It’s sad to say, but these days the only sure fire way to avoid wasting your time with bait-and-switch promises of fully local operation is to flat out ignore all products with WiFi interfaces. It seems like you’re only safe with Zigbee, Z-Wave, perhaps Matter over Thread, and maybe Bluetooth (but probably not Bluetooth, because Bluetooth-enabled devices are often intended to be used with a mobile app, which generally invites trouble.)

The “Local Poling” and “Local Push” integration pages should have a mandatory section that describes in detail the steps to commission the device out of the box prior to integrating it with Home Assistant. If the steps involve installing a mobile app or require an Internet connection, that should be disclosed and the integration should be reclassified as something else like “Local Push - Some/All Devices Need Commissioning with Proprietary App/Cloud”. (but preferably more succinct) Pages for such integrations should also warn prospective buyers/users that devices may be rendered unmanageable in the future if the manufacturer discontinues the app/service or pushes an update that breaks local connections.

Another risk is the manufacturer pushing a firmware update through the app at some point in the future which disables LAN control before I can enable it.

All too common. Just look at the recent news about Bambu Lab 3D printers. Avoid!

The required app setup is a big downside to using devices like this. Flashing the internal microcontroller with ESPHome or Tasmota is a permanent solution, but not always possible, and certainly not as straightforward.

I’ve been known to do this, but I honestly find the popularity of the ESP32/ESP82xx platforms a little worrisome from a supply chain standpoint and I also try to limit my use of them for ethical reasons. (that’s a topic for a different thread, though) I look forward to similar options for devices based on the RP2350.

3 Likes
13

There seems to be a state / approach missing that HA could help with. Rather than assuming state, it could be derived from other sensors. For example, time of day and Lux level could be used to derive the state of a lamp. Sound level could be used to derive if the TV is on or off.

14

It’s not fair statement. Opt in for Shelly and you will be happy.
BTW upcoming 4th generation of Shellies is going to have wifi, bt and zigbee all together on board.

4 Likes
15

IIRC Paulus posted a comment a year ago or so that he would also like to create a device database (similar to the one Zigbee2MQTT) but for all devices compatible with all locally controlled devices that are natively compatible with Home Assistant.

If that ever comes to frutition then I imagine that such a device database could contain detailed information for each product (at least for Zigbee, Z-Wave, Thread, and Bluetooth devices).

I agree. Though I would also add recommendation to stay away from all Matter devices for now as well, and that includes Matter devices that uses Thread, because Matter still have loopholes that allow manufactures to add propriatory features that are only available in their own app and/or hub.

I would also recommend to try avoid all products that state that they require the manufacurer’s own app or hub/gateway/bridge because even if they claim it works locally there is still a chance it requires a clould connection/service at some point or that the manufacturer push a firmware update that changes it (like what is happening to Bambu Lab’s 3D pinters right now).

1 Like
16

Sorry. The point I was trying to make is that not all but most WiFi-based products are that way, and if you find yourself looking at mass market “smart” devices on a typical retailer’s shelf, ignoring the ones with a WiFi logo is a way to avoid wasting your time. Obviously, there are many WiFi-based products that don’t have this issue, but they are in the minority.

Thanks for the suggestion of Shelly, but I am avoiding their products for now. I’d be more favorable toward their brand if they didn’t use Chinese micro controllers (ESP32), which concern me on grounds of security, human rights, and democracy. That’s a tangent for a different thread or DMs, though.

1 Like
17

Do you care the same way about your networking stack too?

1 Like