OpenAI Unveils GPT-4o Mini: Revolutionizing Affordable AI
Towards Intelligence Too Cheap to Meter:
- Pricing: 15 cents per million input tokens, 60 cents per million output tokens
- Performance: MMLU score of 82%, blazing fast speed
Introduction of GPT-4o Mini:
OpenAI has introduced GPT-4o Mini, its latest and smallest AI model, which is designed to be more affordable and faster than the current state-of-the-art AI models. OpenAI has announced that GPT-4o Mini is now available for developers and consumers through the ChatGPT web and mobile app, with enterprise access starting next week. This model will replace GPT-3.5 Turbo as the smallest model offered by OpenAI.
Performance Highlights:
GPT-4o Mini excels in reasoning tasks involving both text and vision, surpassing other leading small AI models like Gemini 1.5 Flash and Claude 3 Haiku. According to data from Artificial Analysis, GPT-4o Mini scored an impressive 82% on MMLU, a benchmark for measuring reasoning abilities, compared to 79% for Gemini 1.5 Flash and 75% for Claude 3 Haiku. Additionally, on MGSM, which measures math reasoning, GPT-4o Mini achieved an 87% score, outperforming both Flash and Haiku.
Cost and Efficiency:
GPT-4o Mini is priced at just 15 cents per million input tokens and 60 cents per million output tokens, making it more than 60% cheaper than GPT-3.5 Turbo. With a context window of 128,000 tokens, which is roughly the length of a book, GPT-4o Mini is ideal for high-volume, simple tasks. Independent early tests have confirmed that GPT-4o Mini is more than twice as fast as its predecessors, with a median output speed of 202 tokens per second.
Versatility and Future Capabilities:
Currently, GPT-4o Mini supports text and vision in the API, with plans to include video and audio capabilities in the future. It can handle up to 16,000 output tokens per request and supports multiple languages efficiently.
Safety and Reliability:
OpenAI prioritizes safety in its AI models. GPT-4o Mini incorporates the same safety measures as GPT-4o, utilizing reinforcement learning with human feedback (RLHF) and new techniques such as instruction hierarchy to resist jailbreaks and prompt injections. This ensures that responses from GPT-4o Mini are reliable and safe for large-scale applications.
Availability and Pricing Details:
- Developers: Priced at 15 cents per million input tokens and 60 cents per million output tokens
- ChatGPT Users: Available today for Free, Plus, and Team users, replacing GPT-3.5
- Enterprise Users: Access starts next week
The Future of AI:
OpenAI's commitment to making AI accessible is evident with the release of GPT-4o Mini. This model paves the way for developers to build scalable, powerful AI applications efficiently and affordably. As AI becomes increasingly integrated into our digital experiences, GPT-4o Mini is set to play a crucial role in making these technologies more accessible and reliable.
OpenAI continues to lead the way in AI innovation, driving down costs while enhancing capabilities, and making intelligence broadly accessible to all.
OpenAI's Introduction of GPT-4o Mini:
On Thursday, OpenAI announced its latest AI model, GPT-4o Mini. This new model is both cheaper and faster than OpenAI's current cutting-edge AI models. Designed for developers, GPT-4o Mini is also available through the ChatGPT web and mobile app for consumers starting today. Enterprise users will gain access next week. According to OpenAI, GPT-4o Mini outperforms industry-leading small AI models on reasoning tasks involving text and vision. As small AI models improve, they are becoming more popular among developers due to their speed and cost efficiency compared to larger models like GPT-4 Omni or Claude 3.5 Sonnet. These models are particularly useful for high-volume, simple tasks that developers might repeatedly call on an AI model to perform.
Performance and Cost Efficiency:
GPT-4o Mini will replace GPT-3.5 Turbo as the smallest model offered by OpenAI. The company claims that its newest AI model scores 82% on MMLU, a benchmark for measuring reasoning, compared to 79% for Gemini 1.5 Flash and 75% for Claude 3 Haiku, according to data from Artificial Analysis. On MGSM, which measures math reasoning, GPT-4o Mini scored 87%, compared to 78% for Flash and 72% for Haiku.
Chart Comparing Small AI Models:
According to Artificial Analysis, the following chart compares small AI models:
- GPT-4o Mini: 82%
- Gemini Flash: 79%
- Claude Haiku: 75%
- GPT-3.5 Turbo: 69.8%
- GPT-4o: 88.7%
Further Advantages:
OpenAI states that GPT-4o Mini is significantly more affordable to run than its previous frontier models, and more than 60% cheaper than GPT-3.5 Turbo. Currently, GPT-4o Mini supports text and vision in the API, and OpenAI plans to add video and audio capabilities in the future.
"For every corner of the world to be empowered by AI, we need to make the models much more affordable," said Olivier Godement, OpenAI’s head of Product API, in an interview with TechCrunch. "I think GPT-4o Mini is a really big step forward in that direction."
Pricing for Developers:
For developers building on OpenAI’s API, GPT-4o Mini is priced at 15 cents per million input tokens and 60 cents per million output tokens. The model has a context window of 128,000 tokens, roughly the length of a book, and a knowledge cutoff of October 2023.
Size and Speed:
OpenAI has not disclosed the exact size of GPT-4o Mini but said it is roughly in the same tier as other small AI models like Llama 3 8b, Claude Haiku, and Gemini 1.5 Flash. However, the company claims that GPT-4o Mini is faster, more cost-efficient, and smarter than industry-leading small models based on pre-launch testing in the LMSYS.org chatbot arena. Early independent tests seem to confirm this.
“Relative to comparable models, GPT-4o Mini is very fast, with a median output speed of 202 tokens per second,” said George Cameron, Co-Founder at Artificial Analysis, in an email to TechCrunch. “This is more than 2X faster than GPT-4o and GPT-3.5 Turbo and represents a compelling offering for speed-dependent use cases, including many consumer applications and agentic approaches to using LLMs.”
New Tools for ChatGPT Enterprise:
In addition to the release of GPT-4o Mini, OpenAI announced new tools for enterprise customers on Thursday. In a blog post, OpenAI introduced the Enterprise Compliance API, which helps businesses in highly regulated industries like finance, healthcare, legal services, and government comply with logging and audit requirements.
Enterprise Compliance API:
OpenAI says these tools will allow administrators to audit and take action on their ChatGPT Enterprise data. The API will provide records of time-stamped interactions, including conversations, uploaded files, workspace users, and more.
Granular Control for Workspace GPTs:
OpenAI is also giving administrators more granular control for workspace GPTs, a custom version of ChatGPT created for specific business use cases. Previously, administrators could only fully allow or block GPT actions created in their workspace. Now, workspace owners can create an approved list of domains that GPTs can interact with.
Making AI Broadly Accessible:
OpenAI is committed to making intelligence as broadly accessible as possible. Today, the company announced GPT-4o Mini, its most cost-efficient small model. OpenAI expects that GPT-4o Mini will significantly expand the range of applications built with AI by making intelligence much more affordable. GPT-4o Mini scores 82% on MMLU and currently outperforms GPT-4 on chat preferences in the LMSYS leaderboard. It is priced at 15 cents per million input tokens and 60 cents per million output tokens, an order of magnitude more affordable than previous frontier models and more than 60% cheaper than GPT-3.5 Turbo.
Broad Range of Tasks:
GPT-4o Mini enables a broad range of tasks with its low cost and latency. These include applications that chain or parallelize multiple model calls (e.g., calling multiple APIs), passing a large volume of context to the model (e.g., full code base or conversation history), or interacting with customers through fast, real-time text responses (e.g., customer support chatbots).
Text and Vision Support:
Currently, GPT-4o Mini supports text and vision in the API. In the future, it will support text, image, video, and audio inputs and outputs. The model has a context window of 128,000 tokens, supports up to 16,000 output tokens per request, and has knowledge up to October 2023. Thanks to the improved tokenizer shared with GPT-4o, handling non-English text is now even more cost-effective.
Superior Textual Intelligence and Multimodal Reasoning:
GPT-4o Mini surpasses GPT-3.5 Turbo and other small models on academic benchmarks across both textual intelligence and multimodal reasoning. It supports the same range of languages as GPT-4o and demonstrates strong performance in function calling, enabling developers to build applications that fetch data or take actions with external systems. GPT-4o Mini also offers improved long-context performance compared to GPT-3.5, especially for language generation tasks.
Performance on LMSYS.org Leaderboard:
Pre-launch testing in the LMSYS.org chatbot arena showed GPT-4o Mini performing at par with GPT-4o on chat preference tests, reflecting overall model quality. The model ranks first on the LMSYS.org leaderboard for small models. GPT-4o Mini's median response time is approximately 202 tokens per second, providing over 2X improvement in latency compared to GPT-4o and GPT-3.5 Turbo.
Advanced RLHF Techniques:
To ensure the safety and alignment of GPT-4o Mini, OpenAI uses the same RLHF (reinforcement learning with human feedback) techniques that power GPT-4o. In addition to RLHF, OpenAI is applying new methods like instruction hierarchy to GPT-4o Mini, making it more resistant to jailbreaks and prompt injections. This ensures that GPT-4o Mini provides reliable, safe responses for a wide variety of applications.
Customer Testing and Enterprise Rollout:
Since January, OpenAI has been working with early customers to test GPT-4o Mini and build features that meet the needs of large enterprises. These features include tools that improve enterprise security, audit logging, and administrative controls. Starting today, GPT-4o Mini is available for Free, Plus, and Team users of ChatGPT. Next week, enterprise customers will gain access to the model in ChatGPT Enterprise.
Pioneering the Future of AI:
OpenAI’s GPT-4o Mini marks a significant milestone in the development of small AI models. By combining cost efficiency, high performance, and advanced safety measures, GPT-4o Mini is poised to revolutionize the AI landscape. Its introduction is set to empower developers and businesses alike, making AI technologies more accessible and practical for a wide array of applications. OpenAI’s ongoing commitment to innovation ensures that GPT-4o Mini will play a pivotal role in the future of AI, driving forward the integration of intelligent systems into everyday digital experiences.
