How to use regex to change display names of all entities in my auto-entites card?

You will have to use a “template” option of auto-entities card.
Example.
Untested:

    filter:
      template: >-
        {% for state in states.sensor | selectattr('entity_id','search','temperature') -%}
          {{
            {
              'entity': state.entity_id,
              'name': state.attributes.friendly_name.split('RHT Sensor Temperature (')[1].replace(')','')
            }
          }},
        {%- endfor %}