Claude 3.5 Sonnet, Anthropic's new model, has outperformed GPT-4 in numerous assessments.
Claude 3.5 Sonnet outperforms both Anthropic’s Claude 3 Opus model and its competitors on the market.
OpenAI’s competitor Anthropic has introduced a new large language model, Claude 3.5 Sonnet. Positioned as the first in the new 3.5 series built upon the Claude 3 announced in March, Claude 3.5 Sonnet outperforms both Anthropic’s Claude 3 Opus, introduced about three months ago, and its competitors on the market.
Key Features of Claude 3.5 Sonnet
According to the information shared by the company, Claude 3.5 Sonnet operates at twice the speed of Claude 3 Opus. This allows Claude 3.5 Sonnet to handle complex tasks such as providing context-sensitive customer support and automating multi-step workflows. The model excels in numerous evaluations, surpassing its competitors in assessments such as GPQA (graduate-level reasoning), MMLU (undergraduate-level knowledge), and HumanEval (coding proficiency).
Code Writing
Claude 3.5 Sonnet achieved a 92% performance in zero-shot code evaluation, outperforming OpenAI’s GPT-4 with 90.2% and Google’s Gemini 1.5 Pro with 84.1%. According to Anthropic, given the appropriate tools, the model can independently write, edit, and execute code with reasoning capabilities surpassing Claude 3.0. This enables developers to easily transition between different codebases and update legacy applications to new frameworks using Claude 3.5 Sonnet.
Vision Model
The model also features Anthropic’s most advanced vision model developed for reasoning. Thus, we can say that Claude 3.5 Sonnet has acquired foundational abilities to understand and interpret written language and symbols. The model can quickly comprehend and respond to complex mathematical problems, charts, and graphs that users write. It’s worth noting that the model can even transcribe text from flawed images.
Supporting requests up to 200,000 tokens, Claude 3.5 Sonnet shares the same context size as the Claude 3 model family. The model costs $3 per million input tokens and $15 per million output tokens. According to information provided by the company, Claude 3.5 Sonnet emerges as a more optimal choice compared to the Claude 3 Haiku and Claude 3 Opus models.
The new feature Artifacts
With this release, the company has introduced a new feature called Artifacts on the web interface Claude.ai. As part of this feature, a dedicated window is added next to the chat box of the chatbot. When you ask Claude to create code, text content, or website designs, this area will transform into a workspace. This allows you to see a real-time representation of the action you requested, and you can edit and enhance this representation as needed.
The Artifacts workspace is currently in the preview stage. The company aims for users to collaborate with Claude through this workspace. Previously, users had to copy the provided code or website designs to another interface or editor when requesting a code snippet or website design. This new feature eliminates that need, allowing users to interact directly with the model.
Claude 3.5 Sonnet can be accessed for free via Claude.ai and the Claude iOS app. Subscribers to Claude Pro and Team plans have access to the model with higher limits. Additionally, Anthropic mentions that the model can be used through the application programming interfaces of Amazon Bedrock and Google Cloud’s Vertex AI platform. The company has announced plans to release Claude 3.5 Haiku and Claude 3.5 Opus models to complete the Claude 3.5 model family by the end of this year.
