Google Assistant Setup Issue - "Could not update the setting. Please check your connection"

To anyone experiencing this issue I appear to have found one of the causes for the “Could not update the setting. Please check your connection” error. It seems certain entities in home assistant will cause this error to appear even if there isn’t a connection problem. To eliminate this being a cause set expose by default to false then specify individual entities to expose. Examples below:

google_assistant:
  project_id: !secret project_id
  client_id: !secret assistant_id
  access_token: !secret assistant_token
  expose_by_default: false

.

customize:
  light.light_name:
    google_assistant: true
    google_assistant_name: light one