EmbeddingGemma 2 repurposed for speech-to-text
Researchers transformed Google’s EmbeddingGemma 2, a multimodal embedding model, into a speech-to-text system. The model processes an hour of audio in about 13 seconds on a base MacBook Air, with all processing done on-device. The conversion involved keeping the audio encoder unchanged while adding a small reader to map audio slices to text space, selecting the closest English word. A secondary model adds punctuation and capitalization. The system was trained for under two hours using openly licensed English speech data, with the base model never specifically trained for speech-to-text tasks. Performance is slightly lower than leading speech models but shows promise for future improvements.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.