Revised Estimates Suggest Higher Expenses for Sophisticated AI Tasks
- OpenAI’s o3 “reasoning” AI model may be more expensive to run than initially estimated, with costs potentially reaching $30,000 per task.
- The Arc Prize Foundation has revised its computing cost estimates for o3, citing the model’s high computational requirements and test-time compute usage.
- The pricing of o3, when released, is expected to be high, potentially rivaling or exceeding the costs of human contractors or staffers for certain tasks.
The recent update to the Arc Prize Foundation’s estimates for OpenAI‘s o3 model has sparked interest in the true cost of running advanced AI models. The o3 model, unveiled in December, was showcased alongside the ARC-AGI benchmark, designed to test highly capable AI. Initially, the estimated computing cost for the best-performing o3 configuration, o3 high, was around $3,000 per ARC-AGI problem. However, the Arc Prize Foundation has since revised this estimate to a much higher figure, potentially around $30,000 per task.
This revision highlights the significant expenses associated with running today’s most sophisticated AI models, particularly in the early stages. OpenAI has yet to release or price o3, but the Arc Prize Foundation believes that the pricing of OpenAI’s o1-pro model, the company’s most expensive model to date, can serve as a reasonable proxy. According to Mike Knoop, co-founder of the Arc Prize Foundation, o1-pro is a closer comparison due to the amount of test-time compute used by o3.
The high computational requirements of o3 high are a significant factor in its potentially high cost. The model reportedly used 172 times more computing resources than o3 low, the lowest-computing configuration of o3, to tackle ARC-AGI tasks. This excessive computational usage is not surprising, given the model’s need for 1,024 attempts at each task to achieve its best score. As AI researcher Toby Ord pointed out, this inefficiency may offset the cost advantages of using AI models over human contractors or staffers.
Rumors have been circulating about OpenAI’s plans to introduce pricey plans for enterprise customers, with reports suggesting that the company may charge up to $20,000 per month for specialized AI “agents.” While some may argue that even OpenAI’s priciest models will be more cost-effective than human labor, the true cost of running advanced AI models like o3 remains to be seen. As the AI landscape continues to evolve, it is essential to consider the potential expenses associated with developing and deploying sophisticated AI models.
The o3 model’s high cost is a reminder that the development and deployment of advanced AI models are complex and resource-intensive tasks. As AI technology advances, it is crucial to balance the benefits of these models with their associated costs. The pricing of o3, when announced, will likely have significant implications for the adoption and use of advanced AI models in various industries.