Groq Revolutionizes AI Querying with Llama Models and Whisper Integration

Last week, the company quietly launched this feature, allowing you to enter queries either by typing or speaking. For voice queries, Groq uses OpenAI’s open-source automatic speech recognition and speech translation model, Whisper Large V3, to convert your voice into text. The converted text is then added as a prompt for the large language model. The resulting outputs are said to be much faster and smarter than those previously offered by the company.

The engine on Groq’s site defaults to using Meta’s open-source Llama3-8b-8192 large language model. The engine also uses the larger Llama3-70b model, released by Meta in April, and allows you to choose from some Gemma and Mistral models. According to the company, Groq will soon support other models as well.

Use of LPU Instead of GPU

Groq’s value proposition is to perform AI tasks much faster and more cost-effectively than its competitors. The company achieves this value proposition through its language processing unit (LPU). Because LPUs operate linearly, they outperform GPUs in these types of tasks.

According to shared information, Groq’s technology uses about one-third of the power of a GPU. Recall that Google’s carbon emissions increased by approximately 50% due to AI-driven energy demand. In this context, Groq’s efficiency stands out as a notable alternative in the AI space, which is typically dominated by GPUs.

According to Groq CEO Jonathan Ross, Groq has been offering its services for free for about four months and is currently used by over 282,000 developers. Groq provides a console for developers to build their applications. It’s worth noting that developers creating applications on OpenAI can quickly transition their applications to Groq.