Highly likely this needs to be done via card-mod. I only advanced a bit as css is not my thing.
As the extrema label is a class I could identify it easily but the box of the label is not static (no class) and I (no css expert, repeating) have no clue how to identify the <rect id=“Svg…”
Youcan have a look here
https://community.home-assistant.io/t/card-mod-add-css-styles-to-any-lovelace-card
EDIT: it seems to work afterall, required a screen-refresh (ctrl-5) and in edit-mode it is a bit iffy 
