Observing a new user for 3 hours: Automation UI improvements

I am an experienced HA user but I had the pleasure of helping and observing a new HA user create simple light automations for 3 hours this weekend. From that I have identified several pain points in the automation UI and have suggestions for how to improve them. I am posting these here for input/discussion.

Cheers,
-David

Problem 1: Choose entity pop-up requires repeated searching for the same items.
Proposal 1: List the 5 most recently used entities at the top of the pop-up
(Most frustrating by far: this made the experience go from “oh cool” to “arg, not again”.)

The most frustrating part of using the UI by far was having to search for “bedroom light strip” dozens of times to create 10 automations. If the pop-up list remembers the last 5 most recently used items at the top, this would have made the process of creating automatons immensely more enjoyable.

This was most frustrating in simple things like if/then/else blocks where the same entity needed to be searched for 2 or 3 times in creating just one automation.

Problem 2: The Add Action menu has unclearly-named items organized alphabetically
Proposal 2: Clarify the names, re-order by common usage, and group by function
(The new user would not have figured this out without my help)

Here is a simple problem the new user ran into: To figure out how to turn a light on or off you have to know that the item is “Call service” and then you search for a “service” called “light.turn_on” or “home assistant.turn_on”. This is not intuitive unless you are used to writing YAML and using “service: “ in your code.

Further, the items are sorted alphabetically, which means that you have to look through the whole list to find items by function and the very uncommonly used by beginners “Condition” is the third item they see. For example, choose, if-then, and repeat are all control-flow items, so they should be together so you can see the options for control flow, while call service, scene, and play media are all actions that should be together.

In addition, “device” is unclear: what does “device” do? When I select it, I see that it provides common actions for most devices. This is probably the most useful item in the list for new users, but they will only discover it by accident since it is the 5th one down and not clear what it means.

Here is an alternative organization that will make the menu easier for new users by 1) clarifying the names, 2) putting the most common ones at top, and 3) grouping the items by what they do. In particular, this provides 4 example services so that users can see what the most common services look like and see how to explore other services.

  • Common Device Actions <— (renamed from “Device” to say what it does)
  • Activate Scene
  • =======
  • Simple Turn On <— (inserts a homeassistant.turn_on service call)
  • Simple Turn Off <— (inserts a homeassistant.turn_off service call)
  • Simple Toggle <— (inserts a homeassistant.toggle service call)
  • Advanced Light Turn On (color/brightness) <— (inserts a light.turn_on service call)
  • Play Media
  • Other Actions… <— (inserts a service call, but renamed to make it clear what that is)
  • =======
  • If-then
  • Repeat
  • Choose (multiple If-then) <— (explain what this does in the title)
  • Stop Condition <— (explain what “Condition” really does)
  • ======= <— (special/advanced items at the bottom)
  • Generate Event
  • Define Variable
  • Run in parallel
  • Wait for a template
  • Wait for a trigger

Problem 3: Searching for services does not find names that do not match YAML
Proposal 3: Ignore “.” “_” “:" and “ “ in searching the add service pop-up
(This problem occurred every single time the new user went to add a service because you can only remember the “.” and “:” if you are coming from YAML.)

If you search for the service “light turn on” in the add service pop-up you find nothing. You have to search for “light.turn_on”. This is a problem for new users, as they have no reason to use either . or _ in their search. The search should ignore those characters so “light turn” will find “light.turn_on” and “light.turn_off”.

Problem 4: light.turn_on is extremely complicated if you just want to set the brightness
Proposal 4: Hide most of the advanced functions unless the user chooses to show them
(The new users got lost in the list of options for light.turn_on each time and was never clear on which of the four brightness options should be used.)

Most new users just want to use light.turn_on to set the brightness or color, so those should be the most visible options. But right now we have 6 color options and 4 brightness options to choose from.

This can be simplified by showing only the common options first, and having the others shown only if the user chooses to show them:

  • Target
  • Transition
  • Color
  • Color temperature
  • Brightness
  • Effect
  • Advanced color/brightness <— (hidden by default)
    • RGBW-color
    • RGBWW-color
    • Color name
    • Hue/Sat color
    • XY-color
    • Color temp (Kelvin)
    • Brightness value
    • =======
    • Brightness step value
    • Brightness step
    • Profile
    • Flash

Problem 5: It is easy to get lost in the automation list when there are if/then or choice blocks
Proposal 5: Color each depth level the same (and increasingly dark) shade
(The new users was constantly scrolling up and down to try and figure out where they were in if/then blocks and choose blocks. The worst was adding an OR block which made the if/then even harder to follow.)

This can be solved by using increasingly dark shades of gray (or alternating) as background colors for each level of depth in the hierarchy. This would make sure everything in the “if" part was visually separated from the "then” part and that the items in the “or” part in the “if” part were visually grouped. (Clearly there are problems with how many levels deep and how dark, but doing this for just the first 4 levels would probably cover 99% of new users.)

Problem 6: Not clear if you should use a Device or an Entity as a target
Proposal 6: Hide one or the other
(Low-priority: the new user just always used entity, but did not know why.)

It is not clear why you should choose a device or an entity when applying a service call. Both are available and most things are listed in both. If one is preferred, it should be shown and the other hidden.

Problem 7: The Create helper dialog calls “input_select” a “Dropdown” but the call service menu has no service for “dropdown”.
Proposal 7: Searching for “dropdown” in the call service menu should find “input_select”.
(Low-priority: It’s impossible to write automations for “Dropdowns” for new users as they do not know they have to search for “input_select”.)

Those are really good observations. Even tho i dont use the ui I agree with almost every one of them and can see where you are coming from.

The only one that I gave reservations about is using device triggers/conditions/actions as the preferred option. But that is a philosophical opinion.

Device logic has the huge downside that if the device you are using fails and needs replaced then there is no easy way to replace the device in your automations, etc. The only way to do it is to manually edit every automation, etc where its used to use the new device id.

If you use entities instead you just need to update the new device entity_id’s to be the same as the old one and everything else just works.

1 Like

A nice constructive post

I agree with every item on the list except Devices should be hidden by default and entity states should be made easier to use for the reasons @finity mentions.

Someone does software BA work… :wink:

1 Like

Really constructive ideas! :slight_smile: :+1:

As others said as well, but yet nobody, even you @dbs, voted for this!? :wink: Don’t forget to vote yourself!

2 Likes

Can you qualify “new”?

For example, did the user read the documentation, at least the basic concepts of an automation, before diving into the Visual Automation Editor? Or did they avoid/skim it and dove in head first? :slightly_smiling_face:

BTW, honest question; I’m not being defensive about the Visual Automation Editor. It’s currently better than it’s ever been but there’s room for improvement.

FWIW, here’s what a new openHAB user encounters when creating a basic Rule.

1 Like

@123 Sure! This was a user who had never used HA before. The user had created a description of what they wanted each of their automations to do, and I sat down with them to work through how to make them.

This user had not read any documentation. (I suspect that is the typical case, and it is certainly the one the UI should aim for supporting.) Reading documentation would have explained that “service call” is the HA terminology for “doing something” and it would have have helped with the first time knowing you need to search for “light.turn_on”. However, the problems I listed above occurred not just on the first automation, but even on the 5th and 6th ones, when the user was no longer “new.” For those frustrations I doubt the documentation would have helped.

2 Likes

Have they ever read about or used openHAB? If not, sit them down with openHAB and create the same automations.

I predict their experience will be similar to using Home Assistant. Both are complex applications that require some familiarity with their terminology and concepts.

That’s a known path to failure/frustration for many things, not just software applications. There’s a reason for the acronym RTFM.

1 Like

Thank you!!!

To sit down and watch a new user, then document the pain points, is a real contribution.

I know software companies (like Micro$oft) used to do this back in the day. From what I’ve seen, this sort of user focus has become uncommon as coding is outsourced to the low bidder. And the quality of their products has suffered for it.

In the case of open-source software, there can sometimes be push-back against constructive comments like this. Hopefully your recommendations are seriously considered.

1 Like

@123 You are correct in both regards: not reading tutorials makes things harder and other software is hard to use too. But I think we can make a few simple changes to eliminate nearly all the frustrations this new user experienced. That’s a good thing for both new users (regardless of whether they read tutorials) and for everyone else too!

2 Likes

It contributes to the existing body of knowledge that if you attempt to use complex software without first learning its concepts and terminology, the outcome is typically high frustration and low productivity.

Again, I am not suggesting that there’s no room for improvement but the outcome of this experiment would have been more surprising if the user had breezed through the creation of an automation. As it stands, their difficulties were predictable.

1 Like

I believe there is a balance, and the development of HA has been constantly pushing this balance away from having to understand the complexity. My comments are about how we can move it significantly further for little effort. I believe that is worthwhile.

Indeed, I was able to get a completely new user to do some complex automations with only a little help. That is an amazing testament to much easier HA has become over the past 5 years. If we can fix these pain points, I doubt that new user would have even needed my help, which would be amazing!

When I first started using HA there was no UI for any of these things. Everything had to be written by hand in YAML. (And the whole configuration had to be re-loaded each time and you could break it so HA wouldn’t reload!) That was very difficult, and, as a result, there were few people using HA.

But each time new UIs are added or updated the goal is to reduce the amount of complexity users need to know. HA is incredibly complex, but the UI lets you use most of it without knowing most of the complexity. We should strive for the most easy-to-use UI possible, and we should strive to enable people to use it without having to figure out the complexity.

Now that is super-hard. But still worth doing, in my opinion. :slight_smile:

1 Like

I suggest you break these out into individual feature requests. And please search before you create them all, I know a few are duplicate requests.

Thank you.

I have already done so on GitHub (see the links). I did search there, but could not find obvious duplicates. If you know of any please let me know and I will update the GitHub links appropriately!

GitHub isn’t the place for feature requests, not sure how far that will get you. My comment was about this single large feature request.

Oh! I put it in here, under home-assistant - frontend - Discussions - “Share ideas for new features in the automation editors.” There I broke it down into the 7 issues I listed here. Is that not the right place to put it? If so, where, and how should it be presented?

Ah, yes, that’s a good place as long as you had a full proposal for each separate bullet with a lot of detail.

Thanks! Please let me know what I should do to be most helpful. If you think it would be good for me to break these out here too I can do that, but I don’t want to overwhelm people.
-David

In general, when posting feature requests, you should post 1 change per feature request. Nesting 10 changes in 1 request is a recipe for low votes. There is no rule against it, so it’s ultimately up to you.

RTFM is a completely outdated concept, unless you’re specifically referring to a professional / technical / geek / developer type of user. The type of non-technical consumer that HA is trying to target these days (we’re still trying to target them, right ? Because HA’s intended target audience seems to shift around all the time too) is not reading manuals. You can’t make them read manuals. That’s a lost cause. Maybe they watch tutorial videos (and they will fast scroll through the important parts because they are perceived as boring). But RTFM these users and you have instantly lost them.

Pretty much all consumer centric apps these days must be self-explanatory and fully discoverable, specifically to make users able to use them without a manual.

I don’t think OpenHAB is even open for comparison here. We’re competing against the simplicity / complexity of Google Home / Hue / Amazon / Homekit / etc. Solutions that target non-technical users (with varying levels of success).

Oh and @dbs, excellent UX break down on an actual realworld case. I wish we would get consumer usage analysis reports of that level of detail at work.

3 Likes