Whisper JAX to use TPU for (up to) 70x faster results?

Configuration Voice Assistant
1

Have there been any efforts to incorporate the JAX version of Whisper?

It appears to be able to use a TPU (many HA users have Coral TPU to run Frigate inference) instead of a GPU to increase speed.
It interests me as I’ve been using HA with Whisper-Lite on a dedicated core i5 and response takes 7-10 seconds, making if fairly frustrating to use (and significantly decreasing the Wife Acceptance Factor).
Full disclosure: I’m not an AI developer - and so am very grateful to the work done by the programmers and developers making these tools and then integrating them in to HA for us mere mortals.

2

A tpu v4 is not a Coral TPU its one of Googles High end Cloud TPU’s that currently go to v8 Preços  |  Cloud TPU  |  Google Cloud

3

As i was looking exactly for this topic i’d like to rephrase OP’s initial question a bit:

Is it in general possible to use a “consumer-grade” Coral TPU like this one for example: M.2 Accelerator with Dual Edge TPU | Coral.

Apperently, they are quite affordable, and if there is a chance to improve the speech recognition with whisper I’d take it.

4

explained here