Very cool new feature. But I do not understand well the procedure on how (where) to memorize pictures of persons we want HASS to recognize … any tutorial?
1 Like
is a bit complicate for me … sorry not too intelligent 
My goal is to have actions based on recognition of persons
In my family we are 2, so I wouls like to have notificstions for 2 persons and others (person NOT one of the 2).
So to have like
James
Anna
Others
how to do??
in configuration.yaml, in column 1 write this?
service: microsoft_face.create_person
data:
name: 'James'
name: 'Anna'
then what? send this command???
$ curl -v -X POST “https://westus.api.cognitive.microsoft.com/face/v1.0/persongroups/{GroupName}/persons/{Anna}/persistedFaces”
-H “Ocp-Apim-Subscription-Key: YOUR_API_KEY”
-H “Content-Type: application/octet-stream” --data “@/tmp/Anna.jpg”
and then what??
in configuration should be like this?
#MICROSOFT FACE RECOGNITION
microsoft_face:
api_key: xxx
#
service: microsoft_face.create_group
data:
name: 'Family'
#
service: microsoft_face.create_person
data:
group: family
name: 'Anna'
#
image_processing:
- platform: microsoft_face_identify
group: family
source:
- entity_id: camera.entrance
1 Like
@anon35356645
what about the curl command?
Is that a command that must be run one time to register individual?
Not clear to me
Im lost also, although claudio managed to get further than I did. the weblink to the azure website on the home assistant guide is not in english.
I created an account but I dont think I have done it properly, I somehow signed up for a trial account with some credit, so I think I got it all wrong.
I also could do with a more laman step by step guide.
It seems Im not that intelligent either.
Hi !
I’ve tried but the API call have changed :
Setup failed for microsoft_face: Integration failed to initialize.
16:41:15 – (ERREUR) setup.py
Can't load data from face api: Invalid request has been sent.
16:41:15 – (ERREUR) microsoft_face
Error 403 microsoft face api https://francecentral.api.cognitive.microsoft.com/face/v1.0/persongroups
16:41:15 – (ATTENTION) microsoft_face
Same here 
It seems that no one really uses this (low number of hits when searching for MS face).
Now I know why…
There is not a lot you can do wrong here (at least if you just want to let is basically run).
Setup failed for microsoft_face: Integration failed to initialize.
14:20:32 – (ERROR) setup.py
Can't load data from face api: Invalid request has been sent.
14:20:32 – (ERROR) microsoft_face
Error 403 microsoft face api https://germanywestcentral.api.cognitive.microsoft.com/face/v1.0/persongroups
14:20:32 – (WARNING) microsoft_face
Another point what puzzles me is the number of MS Face results on home-assistant.io:
Microsoft Face - Home Assistant (home-assistant.io)
Microsoft Face Detect - Home Assistant (home-assistant.io)
Microsoft Face Identify - Home Assistant (home-assistant.io)
So is Face the base? Do I need all of them?
It’s not clear to me (and not well documented) what’s really needed for face recognition.
I don’t want to detect faces unless it’s needed for recognition.
I want to use it, but i am on the same issue
it is personally for me to complicated, but i am a beginner one and uses most of the time the standard functions and configuration.yml just copy paste and crossing fingers for working.
but if anyone has a good tutorial it would be a pleasure to use this integration.
The integration needs to be updated (this file specifically) to accept a hostname being entered as the endpoint.
Do you mean the endpoint hostname from azure? Like https://.cognitiveservices.azure.com/?
I’ve also tried to integrate the microsoft face service but with the same errors.
Correct. Instead of being a static URL in the code, it needs to accept an input either via integration setup or in configuration.yaml
This is what I think it should look like, with the key changes being:
- Removal of region as configuration input and within the functionality
- Adding configuration input for the endpoint
- Replacing reference to endpoint that uses region, to the configurable input endpoint
I’d love it if someone with access to the repo and better coding knowledge could validate my changes and submit a PR.
"""Support for Microsoft face recognition."""
from __future__ import annotations
import asyncio
import json
import logging
import aiohttp
from aiohttp.hdrs import CONTENT_TYPE
import async_timeout
import voluptuous as vol
from homeassistant.components import camera
from homeassistant.const import ATTR_NAME, CONF_API_URL, CONF_API_KEY, CONF_TIMEOUT, CONTENT_TYPE_JSON
from homeassistant.core import HomeAssistant, ServiceCall
from homeassistant.exceptions import HomeAssistantError
from homeassistant.helpers.aiohttp_client import async_get_clientsession
import homeassistant.helpers.config_validation as cv
from homeassistant.helpers.entity import Entity
from homeassistant.helpers.typing import ConfigType
from homeassistant.util import slugify
_LOGGER = logging.getLogger(__name__)
ATTR_CAMERA_ENTITY = "camera_entity"
ATTR_GROUP = "group"
ATTR_PERSON = "person"
DATA_MICROSOFT_FACE = "microsoft_face"
DEFAULT_TIMEOUT = 10
DOMAIN = "microsoft_face"
SERVICE_CREATE_GROUP = "create_group"
SERVICE_CREATE_PERSON = "create_person"
SERVICE_DELETE_GROUP = "delete_group"
SERVICE_DELETE_PERSON = "delete_person"
SERVICE_FACE_PERSON = "face_person"
SERVICE_TRAIN_GROUP = "train_group"
CONFIG_SCHEMA = vol.Schema(
{
DOMAIN: vol.Schema(
{
vol.Optional(CONF_ENDPOINT): cv.string,
vol.Required(CONF_API_KEY): cv.string,
vol.Optional(CONF_TIMEOUT, default=DEFAULT_TIMEOUT): cv.positive_int,
}
)
},
extra=vol.ALLOW_EXTRA,
)
SCHEMA_GROUP_SERVICE = vol.Schema({vol.Required(ATTR_NAME): cv.string})
SCHEMA_PERSON_SERVICE = SCHEMA_GROUP_SERVICE.extend(
{vol.Required(ATTR_GROUP): cv.slugify}
)
SCHEMA_FACE_SERVICE = vol.Schema(
{
vol.Required(ATTR_PERSON): cv.string,
vol.Required(ATTR_GROUP): cv.slugify,
vol.Required(ATTR_CAMERA_ENTITY): cv.entity_id,
}
)
SCHEMA_TRAIN_SERVICE = vol.Schema({vol.Required(ATTR_GROUP): cv.slugify})
async def async_setup(hass: HomeAssistant, config: ConfigType) -> bool:
"""Set up Microsoft Face."""
entities: dict[str, MicrosoftFaceGroupEntity] = {}
face = MicrosoftFace(
hass,
config[DOMAIN].get(CONF_ENDPOINT),
config[DOMAIN].get(CONF_API_KEY),
config[DOMAIN].get(CONF_TIMEOUT),
entities,
)
try:
# read exists group/person from cloud and create entities
await face.update_store()
except HomeAssistantError as err:
_LOGGER.error("Can't load data from face api: %s", err)
return False
hass.data[DATA_MICROSOFT_FACE] = face
async def async_create_group(service: ServiceCall) -> None:
"""Create a new person group."""
name = service.data[ATTR_NAME]
g_id = slugify(name)
try:
await face.call_api("put", f"persongroups/{g_id}", {"name": name})
face.store[g_id] = {}
entities[g_id] = MicrosoftFaceGroupEntity(hass, face, g_id, name)
entities[g_id].async_write_ha_state()
except HomeAssistantError as err:
_LOGGER.error("Can't create group '%s' with error: %s", g_id, err)
hass.services.async_register(
DOMAIN, SERVICE_CREATE_GROUP, async_create_group, schema=SCHEMA_GROUP_SERVICE
)
async def async_delete_group(service: ServiceCall) -> None:
"""Delete a person group."""
g_id = slugify(service.data[ATTR_NAME])
try:
await face.call_api("delete", f"persongroups/{g_id}")
face.store.pop(g_id)
entity = entities.pop(g_id)
hass.states.async_remove(entity.entity_id, service.context)
except HomeAssistantError as err:
_LOGGER.error("Can't delete group '%s' with error: %s", g_id, err)
hass.services.async_register(
DOMAIN, SERVICE_DELETE_GROUP, async_delete_group, schema=SCHEMA_GROUP_SERVICE
)
async def async_train_group(service: ServiceCall) -> None:
"""Train a person group."""
g_id = service.data[ATTR_GROUP]
try:
await face.call_api("post", f"persongroups/{g_id}/train")
except HomeAssistantError as err:
_LOGGER.error("Can't train group '%s' with error: %s", g_id, err)
hass.services.async_register(
DOMAIN, SERVICE_TRAIN_GROUP, async_train_group, schema=SCHEMA_TRAIN_SERVICE
)
async def async_create_person(service: ServiceCall) -> None:
"""Create a person in a group."""
name = service.data[ATTR_NAME]
g_id = service.data[ATTR_GROUP]
try:
user_data = await face.call_api(
"post", f"persongroups/{g_id}/persons", {"name": name}
)
face.store[g_id][name] = user_data["personId"]
entities[g_id].async_write_ha_state()
except HomeAssistantError as err:
_LOGGER.error("Can't create person '%s' with error: %s", name, err)
hass.services.async_register(
DOMAIN, SERVICE_CREATE_PERSON, async_create_person, schema=SCHEMA_PERSON_SERVICE
)
async def async_delete_person(service: ServiceCall) -> None:
"""Delete a person in a group."""
name = service.data[ATTR_NAME]
g_id = service.data[ATTR_GROUP]
p_id = face.store[g_id].get(name)
try:
await face.call_api("delete", f"persongroups/{g_id}/persons/{p_id}")
face.store[g_id].pop(name)
entities[g_id].async_write_ha_state()
except HomeAssistantError as err:
_LOGGER.error("Can't delete person '%s' with error: %s", p_id, err)
hass.services.async_register(
DOMAIN, SERVICE_DELETE_PERSON, async_delete_person, schema=SCHEMA_PERSON_SERVICE
)
async def async_face_person(service: ServiceCall) -> None:
"""Add a new face picture to a person."""
g_id = service.data[ATTR_GROUP]
p_id = face.store[g_id].get(service.data[ATTR_PERSON])
camera_entity = service.data[ATTR_CAMERA_ENTITY]
try:
image = await camera.async_get_image(hass, camera_entity)
await face.call_api(
"post",
f"persongroups/{g_id}/persons/{p_id}/persistedFaces",
image.content,
binary=True,
)
except HomeAssistantError as err:
_LOGGER.error(
"Can't add an image of a person '%s' with error: %s", p_id, err
)
hass.services.async_register(
DOMAIN, SERVICE_FACE_PERSON, async_face_person, schema=SCHEMA_FACE_SERVICE
)
return True
class MicrosoftFaceGroupEntity(Entity):
"""Person-Group state/data Entity."""
_attr_should_poll = False
def __init__(self, hass, api, g_id, name):
"""Initialize person/group entity."""
self.hass = hass
self._api = api
self._id = g_id
self._name = name
@property
def name(self):
"""Return the name of the entity."""
return self._name
@property
def entity_id(self):
"""Return entity id."""
return f"{DOMAIN}.{self._id}"
@property
def state(self):
"""Return the state of the entity."""
return len(self._api.store[self._id])
@property
def extra_state_attributes(self):
"""Return device specific state attributes."""
attr = {}
for name, p_id in self._api.store[self._id].items():
attr[name] = p_id
return attr
class MicrosoftFace:
"""Microsoft Face api for Home Assistant."""
def __init__(self, hass, server_loc, api_key, timeout, entities):
"""Initialize Microsoft Face api."""
self.hass = hass
self.websession = async_get_clientsession(hass)
self.timeout = timeout
self._api_key = api_key
self._server_url = f"https://{CONF_ENDPOINT}"
self._store = {}
self._entities = entities
@property
def store(self):
"""Store group/person data and IDs."""
return self._store
async def update_store(self):
"""Load all group/person data into local store."""
groups = await self.call_api("get", "persongroups")
tasks = []
for group in groups:
g_id = group["personGroupId"]
self._store[g_id] = {}
self._entities[g_id] = MicrosoftFaceGroupEntity(
self.hass, self, g_id, group["name"]
)
persons = await self.call_api("get", f"persongroups/{g_id}/persons")
for person in persons:
self._store[g_id][person["name"]] = person["personId"]
tasks.append(
asyncio.create_task(self._entities[g_id].async_update_ha_state())
)
if tasks:
await asyncio.wait(tasks)
async def call_api(self, method, function, data=None, binary=False, params=None):
"""Make an api call."""
headers = {"Ocp-Apim-Subscription-Key": self._api_key}
url = self._server_url.format(function)
payload = None
if binary:
headers[CONTENT_TYPE] = "application/octet-stream"
payload = data
else:
headers[CONTENT_TYPE] = CONTENT_TYPE_JSON
if data is not None:
payload = json.dumps(data).encode()
else:
payload = None
try:
async with async_timeout.timeout(self.timeout):
response = await getattr(self.websession, method)(
url, data=payload, headers=headers, params=params
)
answer = await response.json()
_LOGGER.debug("Read from microsoft face api: %s", answer)
if response.status < 300:
return answer
_LOGGER.warning(
"Error %d microsoft face api %s", response.status, response.url
)
raise HomeAssistantError(answer["error"]["message"])
except aiohttp.ClientError:
_LOGGER.warning("Can't connect to microsoft face api")
except asyncio.TimeoutError:
_LOGGER.warning("Timeout from microsoft face api %s", response.url)
raise HomeAssistantError("Network error on microsoft face api.")