AI Inference Costs Are Collapsing: What the 90% Drop Means for Every Builder

In March 2023, running GPT-4 cost $30 per million input tokens. Today, equivalent-quality models cost $1-3 per million tokens. Some optimized models cost pennies. The cost of AI inference is in free-fall, and it’s reshaping every assumption about what’s economically viable to build with AI.

This isn’t a gradual decline — it’s a collapse. And it’s happening because of simultaneous breakthroughs in model architecture, hardware, optimization techniques, and competitive pressure. Understanding these trends is essential for anyone building AI-powered products, because the economics that make your project unprofitable today might make it wildly profitable in six months.

The Numbers: How Fast Are Costs Actually Falling?

Let’s trace the trajectory with concrete data from Artificial Analysis, which tracks LLM pricing across providers:

GPT-4 Class Models (Input Tokens, per Million)

• March 2023: $30.00 (GPT-4)
• November 2023: $10.00 (GPT-4 Turbo)
• May 2024: $5.00 (GPT-4o)
• January 2025: $2.50 (GPT-4o optimized)
• April 2026: $1.25 (GPT-4.1)

Claude Class Models

• March 2023: $32.68 (Claude 1)
• March 2024: $15.00 (Claude 3 Opus)
• June 2024: $3.00 (Claude 3.5 Sonnet)
• February 2025: $3.00 (Claude 3.5 Sonnet maintained)
• April 2026: $3.00 (Claude Opus 4.6 at 4x capability for same price)

Budget Models

• GPT-4o mini: $0.15/M input tokens
• Claude 3.5 Haiku: $0.80/M input tokens
• DeepSeek V3: $0.27/M input tokens
• Gemini 2.5 Flash: $0.15/M input tokens

The trend is clear: a 10x cost reduction roughly every 18-24 months for equivalent capability. This is faster than Moore’s Law.

What’s Driving the Collapse

1. Architecture Innovations

The biggest cost driver is model architecture. Key breakthroughs:

Mixture of Experts (MoE): Instead of activating all model parameters for every token, MoE models only activate a subset. DeepSeek V3’s MoE architecture uses 37 billion active parameters out of 671 billion total, achieving GPT-4-class performance at a fraction of the compute cost. This single innovation cut inference costs by 3-5x.

Speculative Decoding: Use a small, fast “draft” model to generate candidate tokens and a large model to verify them. Since verification is parallel (unlike generation), this can speed up inference 2-3x without quality loss.

Quantization Advances: Running models in lower precision (INT8, INT4, even INT2 for some layers) reduces memory and compute requirements. Modern quantization techniques lose minimal quality while cutting costs 2-4x.

2. Hardware Competition

NVIDIA’s dominance is being challenged, driving prices down:

• AMD MI300X: Competitive performance at lower price points, forcing NVIDIA to adjust pricing
• Google TPU v5p: Custom silicon optimized for transformer inference at Google-scale economics
• AWS Trainium2: Amazon’s custom chips offering 30-40% cost savings over GPU-based inference
• Groq LPU: Purpose-built Language Processing Units achieving extraordinary tokens-per-second at lower power consumption

3. Optimization Stack

Software optimizations compound hardware improvements:

• vLLM and TensorRT-LLM: Open-source inference engines that optimize memory management, batching, and GPU utilization, often achieving 2-3x throughput improvements over naive implementations
• KV Cache Optimization: Techniques like PagedAttention reduce the memory overhead of serving long-context requests
• Continuous Batching: Processing multiple requests simultaneously to maximize GPU utilization
• Prefix Caching: Reusing computation for common prompt prefixes across requests

4. Competitive Dynamics

The AI provider market is now fiercely competitive:

• DeepSeek demonstrated that frontier-class models could be trained for a fraction of the assumed cost, pressuring everyone to reduce prices
• Google aggressively prices Gemini models to drive adoption
• Open-source models (Llama, Qwen, Mistral) provide a free baseline that proprietary providers must justify premiums over
• Inference providers (Together, Fireworks, Groq) compete on price-performance, driving margins toward commodity levels

What Falling Costs Make Possible

Every 10x cost reduction unlocks new categories of applications:

At $30/M Tokens (2023)

Only high-value enterprise applications could justify the cost. Each API call was expensive enough that developers carefully optimized prompt length and limited AI interaction.

At $3/M Tokens (2024-2025)

Consumer-facing AI products became viable. Chatbots, AI-assisted writing, code completion, and search enhancement could operate at reasonable unit economics.

At $0.15-$0.30/M Tokens (2026)

AI becomes viable for applications previously considered too cost-sensitive:

• Real-time AI monitoring: Continuously analyzing logs, metrics, and events with LLMs
• AI-powered search for small sites: Every blog, small business, and documentation site can afford AI search
• Bulk content processing: Analyzing millions of documents, reviews, or social media posts
• AI in IoT: Processing sensor data with language models at the edge
• Always-on AI assistants: Background AI that continuously processes and organizes information

At $0.01/M Tokens (Projected 2027-2028)

AI inference becomes essentially free for most applications. Every application will have an AI layer because the cost of NOT having one exceeds the cost of adding one.

Impact on AI Business Models

Falling inference costs are forcing fundamental business model shifts:

The Margin Squeeze

Companies that built businesses on AI API margins are getting crushed. If you’re wrapping OpenAI’s API and charging a premium, your margin shrinks with every price cut. The wrapper business model is becoming unviable unless you add substantial proprietary value.

The Volume Play

Low costs enable volume plays. Applications that were too expensive at $30/M tokens become profitable when costs drop to $0.30/M tokens — but only if you can attract enough users. The competitive advantage shifts from “we can afford AI” to “we have enough users to make cheap AI profitable.”

Open Source Economics

With inference costs falling, the total cost of ownership for self-hosted open-source models is increasingly competitive with API providers. For companies with predictable workloads, self-hosting eliminates per-token costs entirely (after hardware investment).

The Data Moat

As AI capability becomes commoditized, the differentiator is data. Companies with unique, proprietary data can fine-tune cheap models to outperform expensive general models on their specific tasks. The competitive advantage is shifting from model access to data access.

Predictions: Where Costs Go From Here

Based on the current trajectory and announced improvements:

2026-2027

• Frontier model inference drops below $0.50/M input tokens
• Budget models approach $0.05/M tokens
• Edge inference on consumer devices becomes practical for many tasks
• Batch processing costs become negligible for most applications

2028 and Beyond

• Inference costs approach the cost of electricity to run the hardware
• The focus shifts from cost reduction to capability improvement
• AI becomes a utility like electricity — always available, priced by consumption, unremarkable

What Builders Should Do Now

Don’t Over-Optimize for Current Prices

If you’re building complex caching and optimization systems to save on AI costs, consider whether those systems will still be necessary in 12 months. Sometimes the best optimization is waiting for costs to fall.

Design for AI Abundance

Build architectures that assume AI is cheap. Use AI for features you’d currently consider too expensive — real-time personalization, continuous content improvement, proactive error detection. The cost will catch up to your ambition.

Build Data Assets

Collect and organize proprietary data now. As model access becomes commoditized, your unique data becomes the competitive moat. Fine-tuned models on proprietary data will outperform any general-purpose API.

Monitor Multi-Provider Pricing

Don’t lock into a single provider. Use abstraction layers that let you switch between OpenAI, Anthropic, Google, and open-source models as pricing shifts. Today’s cheapest option might not be tomorrow’s.

Consider Self-Hosting

For predictable, high-volume workloads, the math increasingly favors self-hosting open-source models. The upfront investment in hardware is amortized quickly when you’re processing millions of requests.

The Bigger Picture

The collapse of AI inference costs is one of the most consequential economic trends in technology. It’s analogous to the collapse of cloud computing costs in the 2010s, which enabled SaaS, mobile apps, and the entire modern tech stack.

When something goes from expensive to essentially free, the applications that emerge are always surprising. No one predicted TikTok when mobile data became cheap. No one predicted Uber when GPS became free. The applications that will emerge from near-free AI inference are equally unimaginable today.

The builders who understand this trajectory — and build for the cost structures of tomorrow rather than today — will capture the most value. The future of AI isn’t expensive. It’s abundant, cheap, and everywhere.