GPT-4o, the Completely Free Language Model That Practically Turns ChatGPT into a Human, Introduced: Here Are Its Jaw-Dropping Capabilities
OpenAI held its long-awaited event today. During the event, the company introduced its new flagship model, GPT-4o. This model can speak, see, and hear just like a real human.
OpenAI Artificial Intelligence
OpenAI, a pioneer in the artificial intelligence revolution and known for its impressive models, held its long-awaited event today. During the event, the company made significant announcements, including upcoming updates for the ChatGPT chatbot and the GPT-4 language model.
The most notable announcement was the introduction of the company’s new flagship language model, GPT-4o. This model not only outperforms the current GPT but also operates much faster.
GPT-4o can reason through voice, text, and images.
The new GPT-4o model that the company will offer to its users will power the ChatGPT chatbot. Described as much more efficient and advanced than previous versions of GPT, the model will be able to reason through voice, text, and images. According to the announcements, GPT-4o is inherently a multi-modal AI model, meaning it can understand and generate content across voice, text, and images.
There has been a significant improvement in the voice response capability. Users can now have real-time conversations with the model that feel much more realistic and with less delay. According to OpenAI, the model can respond to voice inputs in as little as 232 milliseconds, which is almost as fast as talking to a human. Previously, the average delay in voice mode was about 2.8 seconds.
Additionally, while ChatGPT is responding, you can interrupt and ask it to change its answer. For example, during a live demo at the event, OpenAI executives asked the model to tell a story about a robot. As the model was speaking, they interrupted and requested it to continue in different tones or emotions. ChatGPT instantly made these changes, fulfilling the requests on the spot. You can watch these moments in the video above.
A demo was also conducted showcasing the model’s advanced visual capabilities. The model can “see” and interpret things shown to it through the device’s camera. For instance, in one demo, an equation written on paper was shown to the model, and help was requested for its solution. ChatGPT assisted in finding the solution. When “I love you ChatGPT” was written on the paper, it responded with an emotional voice, just like a human would.
It can perform real-time translation surprisingly well.
OpenAI has just demonstrated its new GPT-4o model doing real-time translations 🤯 pic.twitter.com/Cl0gp9v3kN
— Tom Warren (@tomwarren) May 13, 2024
Another demo at the event showcased the new model’s translation capabilities. OpenAI demonstrated how GPT-4o can perform real-time translations. While Mira Murati spoke in Italian, other OpenAI employees spoke in English. The model was able to translate sentences very quickly and relay them to the other side with almost no delay.
Through the desktop application, it can read your screen and assist with coding.
With the GPT-4o/ChatGPT desktop app, you can have a coding buddy (black circle) that talks to you and sees what you see!#openai announcements thread! https://t.co/CpvCkjI0iA pic.twitter.com/Tfh81mBHCv
— Andrew Gao (@itsandrewgao) May 13, 2024
In another live demo of GPT-4o, some of its coding abilities were showcased. In this demo, presented through ChatGPT’s new desktop application, the model was able to understand and analyze written code. Additionally, it could also explain what it saw.
GPT-4o can look at you through the camera and make comments.
Say hello to GPT-4o, our new flagship model which can reason across audio, vision, and text in real time: https://t.co/MYHZB79UqN
— OpenAI (@OpenAI) May 13, 2024
Text and image input rolling out today in API and ChatGPT with voice and video in the coming weeks. pic.twitter.com/uuthKZyzYx
After the event concluded, OpenAI shared videos showcasing the abilities of GPT-4o. For instance, in one video, we can see the model engaging in a conversation with a person through the front camera. ChatGPT can understand how the person looks through the camera and make comments about their appearance. It can even comment on the environment where the person is located. It wouldn’t be wrong to say that it has become quite difficult to distinguish it from a human.
2. Emotion and face detection pic.twitter.com/01eJ52JLuE
— Lior⚡ (@AlphaSignalAI) May 13, 2024
In another demo, we see that the model can understand the user’s facial expressions and emotional states by looking at their face. It says to them, “You look very happy and cheerful with a smile and a little excitement on your face.”
It even makes sarcastic jokes:
OpenAI's new GPT-4o model being sarcastic 😅https://t.co/ndItPzKhrN pic.twitter.com/uywEPGZ9CO
— Tom Warren (@tomwarren) May 13, 2024
In another example, the model is asked to make sarcastic jokes. GPT-4o can indeed speak in a mocking tone and make sarcastic jokes.
An example of interrupting and getting it to do what you want:
In this video, the model is asked to count to 10. The OpenAI employee in front of it starts counting, then interrupts and asks it to count faster. We can hear that it successfully fulfills all requests, and at one point, it even says “Okay” in a slightly bored tone.
Two GPT-4o are chatting and singing together.
In one video, we see GPT-4o conversing with another GPT-4o. Towards the end of the video, they even sing a duet together.
When the model sees a dog, it reacts like this:
It can serve as the "eyes" for the visually impaired.
In another example, we’re looking at a very useful application where the model’s visual capabilities could be beneficial. Here, the model acts as the eyes for a visually impaired individual by describing their surroundings.
GPT-4o will be available for use by free users as well!
OpenAI also delivered some very exciting news regarding the language model. According to the company’s announcement, GPT-4o will be available for use through the free version of ChatGPT without any payment. However, there will be a message limit. Once this limit is exceeded, it will automatically revert to GPT-3.5. The company stated that the text and image capabilities of GPT-4o are being rolled out starting today. It was also mentioned that the new voice mode will be introduced to Plus users in the future.
