The Pixel 4’s Live Captions can transcribe any audio to text (others too, soon)

One of the coolest new Android features Google announced at I/O this year was the ability to perform real-time captioning on your smartphone. No, not just live captions for video, but rather for any audio that might pass through your phone’s speakers.

We now know that feature will be available at launch with the Pixel 4, and will arrive on the Pixel 3, 3 XL, and 3a later this year.

I was able to try out the feature early, and it’s quite impressive. This is doubly true considering all the processing happens on the device. Google isn’t sending data to the cloud to improve accuracy, meaning the feature won’t use any of your data and will work if you’re completely off the grid.

The captions can be activated in two moves. Simply press one of the phone’s volume buttons, and then tap the button that shows up under the volume slider. The captions will begin immediately with almost no delay relative to the audio. The quality of the captions will, of course, depend on the quality of the audio, and Google acknowledges that the AI isn’t quite good enough to understand all speech (say, that of a babbling toddler).

Still, the feature has obvious uses if you’re someone with a hearing disability; it opens up a world of video content outside of YouTube (which already has live captions) without having to use specialized software. But for everyone else, it can come in handy if you want to watch a video but can’t use or hear the audio for whatever reason – such as if you’re at work or at a restaurant.

It’s not just for Pixels either; Google says it’s working with other manufacturers to bring the feature to their devices. We’ll see exactly how well the feature works as we spend more time with the Pixel 4. Stay tuned for our review soon.