Make a map card in a view with only the entities you want to see.
Then remove map from the sidebar by removing map:
from your configuration.
example config
views:
- id: map
icon: mdi:map
title: Map
panel: true
cards:
- type: map
entities:
- entity: zone.home
- entity: zone.work
- entity: device_tracker.foo
- entity: device_tracker.bar