Stop Optimizing Your AI for Scarcity

For most software teams, inference has been something to conserve. Every model call costs money, so AI products have generally been designed to get useful results with as few calls as possible. Pick a capable model, send it the right context, generate an answer, and move on. That logic gets harder to defend as capable models become dramatically cheaper.

Last week, Google released Gemini 3.7 Flash, positioning it as a workhorse model for coding and agent workloads. The model is designed for multi-step planning, tool use, software engineering, and automation, with introductory API pricing of $0.75 per million input tokens and $3.75 per million output tokens. The release also arrived only weeks after Gemini 3.6 Flash, showing how quickly the cost-performance curve for agent-capable models is moving.

The interesting part for technology companies goes beyond another Gemini release. Falling inference costs can change the architecture of AI products themselves.

A chatbot might call a model once to answer a question. An agent can call one hundreds or thousands of times while completing a single workflow. It may plan a task, search for information, use tools, inspect the result, revise its approach, recover from an error, and verify the final output. Multiply that behavior across thousands of users or several agents working simultaneously, and model usage quickly becomes a meaningful infrastructure expense.

That has forced teams to be selective about where they use sophisticated reasoning. Expensive models are reserved for valuable tasks, verification gets limited, and continuous agent workflows can be difficult to justify economically. Cheaper workhorse models loosen those constraints. Product teams can start asking what they could build if an additional reasoning step, verification pass, or background agent became inexpensive enough to run routinely.

Traditional software engineering usually treats redundant computation as something to minimize. With probabilistic AI systems, redundancy can improve the product.

Imagine an agent preparing an important analysis. One model can produce the answer, but several inexpensive agents could attempt the task independently while another evaluates their outputs. A workflow could retry automatically when confidence is low or send only the hardest part of the problem to a more expensive frontier model.

Those patterns consume more inference, but they can produce more dependable results. As model costs fall, spending additional intelligence on verification and multiple attempts becomes a legitimate architecture decision. The cheapest workflow may not always be the one making the fewest model calls. It could be the one that uses inexpensive reasoning to prevent errors, reduce human review, or complete more work autonomously.

The industry has spent years competing over which company can produce the most capable model. That race continues, but another competition is developing around how much useful intelligence companies can deliver for each dollar of compute.

Gemini 3.7 Flash is one example of that shift. Models designed for high-volume workloads give developers more freedom to decide when frontier intelligence is actually necessary. A customer support workflow might use a cheaper model for classification, research, and drafting before escalating an unusual case. A coding agent might perform routine analysis with a workhorse model and route a difficult architectural decision elsewhere. Background agents could continuously monitor systems or data without requiring premium inference for every step.

Model selection starts becoming dynamic within the same workflow.

This has an important implication for teams building AI products today. Architectures designed around current inference prices may age quickly if capable reasoning keeps getting cheaper.

Teams can prepare by separating workflows from individual model providers and designing routing layers that make it easier to choose models based on task complexity, cost, latency, and reliability. That flexibility allows a product to take advantage of cheaper models without rebuilding the entire system every time the cost-performance frontier moves. It also creates room for more ambitious agent behavior, since parallel execution, verification loops, retries, and continuous background analysis become easier to justify when each additional reasoning step costs less.

The goal is not to consume tokens for the sake of it. The opportunity is to spend inexpensive intelligence where additional computation creates measurable value.

This issue of Redeployed is brought to you by Tecla Labs: As AI inference gets cheaper, companies can rethink how their AI products are built. More affordable models make it practical to route work across different systems, run agents in parallel, add verification, and use more intelligence throughout a workflow. Building these architectures requires engineers who understand AI infrastructure, model orchestration, and production software. Tecla Labs designs and builds AI systems for companies that need this expertise now, so teams can ship flexible AI products without locking themselves into a single deployment model.

Lower model prices do not guarantee lower AI bills. Agents can consume enormous amounts of inference precisely because they are capable of working continuously. A cheaper model running ten times as often can still cost more overall. Parallel agents introduce coordination overhead, verification adds additional calls, and always-on automation can quietly accumulate usage across an organization. Infrastructure, observability, and human review also remain part of the total cost.

Rapid model turnover creates another problem. Google released Gemini 3.7 Flash only weeks after 3.6 Flash. Teams that tightly couple products to individual models could spend increasing amounts of engineering time adapting systems every time a better cost-performance option appears. Flexible architecture becomes more valuable when the underlying models change this quickly.

Model economics are becoming part of systems design. Engineering teams increasingly need to understand how much a complete AI workflow costs, which steps require expensive reasoning, and where cheaper models are sufficient. That means measuring outcomes at the workflow level rather than focusing only on token prices or benchmark scores.

The useful metric might eventually look more like completed work per dollar. A slightly weaker model could produce better economics if its lower cost allows a company to run more attempts, add verification, or deploy agents across workflows that were previously too expensive to automate. That gives engineering leaders another lever beyond simply upgrading to the latest frontier model.

Google's introductory pricing does not tell us exactly where long-term agent economics will settle. It does show how quickly the cost-performance frontier is moving. If that trajectory continues, product teams will have access to increasingly capable reasoning at lower prices. Some of the biggest gains may come from architectures that deliberately consume more of it through parallel agents, verification, retries, routing, and continuous automation.

Software companies spent the first phase of the AI boom learning how to use intelligence efficiently. The next phase may be about figuring out what becomes possible when they can afford to use much more of it.

Connect With Other Technology Leaders

If you want to connect with other technology leaders having real conversations about AI and how it is changing business, check out GILD Curated Circuit.

More to come…

✔️ Garry Tan says founders who ‘tokenmaxx’ on AI agents will be 2 years ahead — Business Insider

✔️ Why AT&T Is Betting Big on Open-Weight AI — The Wall Street Journal

✔️ Beware the token trap: Why saving on inference might put your ADLC at risk — TechRadar