Response from Google to OpenAI: Project Astra announced

Google I/O 2024 has begun, and everything is about artificial intelligence. The search giant has announced a new multi-modal and interactive Project Astra, which seems like a response to OpenAI.

Six years ago at another I/O developer event, Google showcased an AI demo named Duplex and even managed to book a haircut appointment from a barber. After these years, Google recently announced Project Astra. In the shared demo, Project Astra can see everything around it through a phone camera and answer questions about them.

During today’s keynote speech, Google’s DeepMind CEO, Demis Hassabis, expressed that his team is striving to develop universal AI agents that could assist in daily life. Project Astra represents a step towards this goal.

What is Project Astra?

Project Astra appears to be an application with its main interface as a viewfinder. Google defines Project Astra as an advanced visual and speech-sensitive tool and says that this is the future of artificial intelligence assistants.

In the shared demo, we see the person holding the phone taking the device’s camera and Project Astra to various parts of the office and asking questions. In the example shown, the user vocally commands, “Tell me when you see something making noise,” to which the AI, supported by Gemini, responds, “I see a speaker making noise.” Then, the user asks the AI what the thing pointed to in the speaker is, and receives the answer, “That’s a tweeter. It produces high-frequency sounds.”

Google emphasizes in its statement that the video was shot in a single take and in real-time. In the later parts of the video, Gemini is shown identifying and explaining code snippets on a monitor, as well as telling the user which neighborhood they are in based on the view from the window.

The most impressive part comes when the user asks, “Do you remember where I left my glasses?” In the video, nothing was asked about glasses to the AI, and there wasn’t even a pair of glasses in the scene shown through the phone camera at that moment. However, despite this, Gemini responds, “Yes, I remember. Your glasses are next to a red apple.”

The second interesting part is when the user puts on their glasses and sets down the phone. After the user puts on the glasses, the video transitions to the perspective you would see on a wearable device. In this segment, the AI is asked about the diagram on the board, “What can I add here to make this system faster?” Astra responds, “Adding a cache between the server and the database could speed it up.”

How does it work?

For an assistant to be truly useful, it needs to understand and react to the complex and dynamic world just like humans do. It needs to remember what it sees and hears to understand context and take action. Additionally, it needs to be proactive, trainable, and personal so that users can interact with it naturally and without delay.

Google says it has developed prototype tools based on Gemini, continuously encoding video frames, combining video and speech inputs in a timeline of events, and caching this information for faster recall. Google says it builds on leading speech models but also offers a wider range of tonalities. These tools can better understand the context in which they are used and respond quickly during conversations.

When will Project Astra be released to the market?

Honestly, Google didn’t say when Project Astra will be released, or whether it will be released at all. As the name suggests, it’s a project, and things learned during the process will find their way into Google’s services.

Google also hints that these assistants could be used through your phone or glasses in the future. The mention of “glasses” is significant here. We might be looking at the comeback of Google Glass. On the other hand, DeepMind CEO Demis Hassabis said that some of the demonstrated capabilities, like the Gemini application, will come to Google products later this year.

Scroll to Top
sohpet islami sohbetler omegle tv türk sohbet dini chat cinsel sohbet tıkanıklık açma galeri yetki belgesi nasıl alınır yalama taşı bets10 giriş