Xiaomi Cloud Vacuum Map Extractor

Ildar_Gabdullin · October 28, 2021, 12:12am

I am quite a new person in this topic - please tell where I could make a mistake.

Task - show a Roborock S50 map on a floorplan.
Here is a card with a floorplan:

      - type: custom:decluttering-card
        template: decl_plot_plot_2d
        variables:
          - ...
          - MAP_NEW_ELEMENTS:
              - type: image
                camera_image: camera.xiaomi_cloud_map_extractor
                style:
                  top: 50%
                  left: 50%
                  transform: translate(-50%,-50%) scale(1,1)

The flooplan itself is done by some decluttering-card, then the camera image with a Xiaomi cloud map is added as one more image element.
The key points are top, left & scale.

By default the Xiaomi cloud map comes with these attributes:

    map_transformation:
      scale: 1
      rotate: 0
      trim:
        top: 0
        bottom: 0
        left: 0
        right: 0

The result is below:

Then we need to trim & rotate the image.
The element in a red circle is supposed to be moved to the left top corner of the floorplan.

To trim, the following steps are made:

The picture above was opened in Adobe Photoshop.
The picture is cropped, only map is left:

image1920×1127 192 KB
First set image width = 1000px, then measure left & top margins by a “Ruler” tool:
right 302px, left 272px
Then set image height = 1000px, then measure top & bottom margins by a “Ruler” tool:
bottom 308px, top 233px

Then edit the camera properties (with colors changed):

    map_transformation:
      scale: 3 #avoid blurred picture
      rotate: 90 ## counter-clockwise
      trim:
        top: 23 ### i.e. rounded 233/1000
        bottom: 31
        left: 27
        right: 30
    colors:
      color_map_outside: [0, 0, 0, 0]
      color_map_wall: [0, 255, 0, 85]
      color_path: [255, 255, 255, 127]
      color_charger: [246,178,107]
      color_robo: [255, 0, 0]
      color_scan: [0, 255, 0, 30]

And then a Lovelace card:

      - type: custom:decluttering-card
        template: decl_plot_plot_2d
        variables:
          - ...
          - MAP_NEW_ELEMENTS:
              - type: image
                camera_image: camera.xiaomi_cloud_map_extractor
                style:
                  top: 49.2%
                  left: 58.4%
                  transform: translate(-50%,-50%) scale(2.38,2.42)

The result is below:

Questions:

Is it normal that “walls” in the vacuum’s map do not coincide with actual walls? (basically the plot is 99% correct). BTW, this is why I had to use different scaleX, scaleY for scaling.
I realized that the color_scan property stands for “inside” color; it is specified in the Docs that this property is “Areas not assigned to any room (for software with rooms support)”. But also I used the color_map_wall which is “Walls (for software without rooms support)”. So it is “with support” vs “without support”, where could be a mistake?
The main question - what will happen when I start the Roborock again, will it redraw the map? Or will it keep same map, same proportions - i.e. I will not have to tune my pictures once again?

My next step is to add a Xiaomi Vacuum Interactive Map Card on this floorplan. Have no idea what problems I will face then…

3_14 · October 28, 2021, 1:53am

Yes, vacuums are not 100% accurate
It is just a matter of description, it might differ between software versions
It depends on your vacuum, but in most cases it should keep the same map and just update the path (probably only gen 1 vacuums do not save map)

Ildar_Gabdullin · October 28, 2021, 1:57am

Roborock S50.
In Mi Home (iOS) during a new cleaning it displays the previously generated map and the current new path.
I have a doubt since yesterday I did the same “tuning” and lost everything when started a new cleaning.
May be that was a glitch…

Also, what about the whole my approach - was it correct?

3_14 · October 28, 2021, 2:00am

I have the same model and it works like a charm (you have to enable map saving in vacuum’s settings in Xiaomi Home app).

Yes, very nicely done I plan to add possibility to overlay map over any image in future releases of Map Extractor

PS new (veeeeery big) release of the Map Card coming really soon

Ildar_Gabdullin · October 28, 2021, 2:02am

Great, looking forward to it!
Very incredible work!

3_14 · October 28, 2021, 2:08am

Here you have a little sneak peek of a new version of a Map Card

homebot · October 28, 2021, 2:46am

No I don’t everything was added through integrations in HACS. Looks like the manual configuration asks for the vacuum: entry?

3_14 · October 28, 2021, 8:23am

No, it is not related to extractor. Does the map appear sometimes?

Ildar_Gabdullin · October 28, 2021, 2:23pm

After starting a clean the map lost - all my tunings are lost…
The map seems to start rebuilding from a scratch.

In the robot I enabled the “Store the map” feature - I hoped that my map will be saved but it is not, is it normal?
“Rotate” angle was even changed.
Before I started zone cleaning - the map was the same (i.e.did not vanish) but the zone rectangle was displayed on the map even after finishing & HA reboot, could not remove it, was it normal?
The dock was not moved.

Then AGAIN spent some time for “tuning” the map, then started a zone clean (only one room) - and AGAIN the map shifted.

I am officially tired of this. Seems that the map cannot be used along with a floorplan since it gives an erratic angle & shifts…

homebot · October 28, 2021, 3:40pm

No it only shows the blue screen with the “Failed to retrieve map from vacuum” text.

3_14 · October 28, 2021, 9:52pm

@Ildar_Gabdullin
I have “map storing” turned on since day 1 and my map hasn’t moved in 2 years

@homebot
Map has to be uploaded to the cloud by the vacuum and your vacuum seems to be unhappy with a command that tells it do do that

Ildar_Gabdullin · October 28, 2021, 10:06pm

May be there is some small nuance I have not got yet…

What I did ~2 hours ago:

After completion of the last zoned cleaning (when the map was badly scaled & shifted again) I “tuned” again the map (i.e. map is properly positioned with respect to the Picture elements card with my floorplan).
The “tuning” means:

editing the camera map object properties (along with using Configuration → Server controls → YAML configuration reloading → XIAOMI_CLOUD_MAP_EXTRACTOR tool) - particularly rotate, trim;
editing the Picture elements card with my floorplan - particularly left, top, scale.

Then by using the “Lovelace Xiaomi Vacuum Map card” I selected a zone (by mouse), chose “Zoned cleanup” & pressed “Start”.
Then moved to the HA page with floorplan + map (probably pressed F5).
Then I realized that the map shifted again.

Is there any possible reasons of this behaviour?

BTW, I opened Mi Home (iPad) and the previously generated map is used, the path is painted above it. So it seems that the map is memorized in the robot.

homebot · October 29, 2021, 5:14am

Hm I’m not sure why it would have stopped working after working for so long?

Edit: IP changed on vacuum… after adjusting the host IP everything works great.
Thanks for your assistance and for all your hard work on this project.

-Russ

3_14 · October 29, 2021, 8:31am

@Ildar_Gabdullin
I think that you should wait for your vacuum to learn your map (do some full cleanups after enabling map storing), it shouldn’t change afterwards.

@homebot
Well, that explains a lot

ferro · November 8, 2021, 9:32pm

Hey @3_14 , thanks for your greate support here.
I have since a few days the following “hint”: two factor auth required (see logs)

2021-11-08 22_27_56-Window

I understand what Vacuum Map Extractor like to tell me, but logs don’t show me anything. (settings → logs). If I remeber corret, there is no option to activte the two factor auth. in Vacuum Map Extractor. A few days ago, there was everthing fine and no message or problems.

Could you assist me?
Thanks

Tamsy · November 9, 2021, 12:20am

Often a simple HA restart helps and the map shows up thereafter. See post #494.

3_14 · November 9, 2021, 12:44am

You can also check out these issues: #157, #180

Ildar_Gabdullin · November 10, 2021, 7:40pm

I think you are right!
After several cleanups rooms appeared:

Earlier there were no rooms:

And the most important thing - I do not observe misalignment so far!

Here are some rules to provide same mutual alignment between a floorplan & a vacuum’s map:

Do not shift a charging dock.
Enable “Save map” feature (post).
Do not start cleaning by a vacuum’s “hardware” button.
Do not start cleaning by using a vacuum.start service.
Always start cleaning by using a “zone cleaning” (use this card) or a “segment cleaning” feature by calling this service:

service: vacuum.send_command
service_data:
  entity_id: '[[SENSOR_VACUUM]]'
  command: app_segment_clean
  params: [2, 3, 16, 5, 17]

Room numbers may be retrieved from a vacuum’s map attributes.

Piotr, thank you again for the job!

Ildar_Gabdullin · November 15, 2021, 11:46pm

@ 3_14
Piotr, may be you know:
is it possible to use a template for the params option?

service: vacuum.send_command
service_data:
  entity_id: '[[SENSOR_VACUUM]]'
  command: app_segment_clean
  params: [2, 3, 16, 5, 17]

I would like to generate the params option dynamically dependingly on which rooms are selected for cleaning & how many passes are selected for each room.
I know that there is fantastic Vacuum Interactive Map Card - but so far I cannot integrate it to my floorplan and wanted to select rooms from UI.

3_14 · November 16, 2021, 12:00am

Where do you want to achieve it? It should be possible to do in a script