Republicans and Democrats don’t agree on much, but when it comes to generative AI, they’ve managed to find 32 billion points of agreement.
On May 15, a bipartisan group of senators unveiled a roadmap that would have the U.S. government spending $32 billion annually by 2026 to drive AI-related innovation. The U.S. isn’t alone; AI spending in India is growing at an annual average rate of 31.5%, and will exceed $5 billion by 2027. This is all to say nothing of private sector spending — which, if Google, Microsoft, and Meta are any indication (and they are), is even more astronomical.
If you’re an AI startup, or if you’re in charge of an AI research team within a bigger company, it’s a heady moment. The money spigot is open, and the race is on to see who can build the best tech the fastest.
Software veterans should recognize all this as a new take on an old script. A fresh technology bursts on the scene, early adopters realize its competitive edge, and then leaders throw resources at it in hopes of winning the space.
But there’s a fourth part of that script, one that, having cofounded CloudZero, I’m acutely aware of: Those heady exploratory projects will eventually have to turn into profitable features and products, and that spending, which currently lives in the friendly write-off world of R&D, will shift over to the more scrutinized columns of OpEx and factor into COGS.
DevOps folks correctly assert that “Premature optimization is the root of all evil.” I’m not suggesting that dev teams start optimizing their AI applications now, but I am suggesting they get out in front of the cost nightmare that tends to follow periods of high innovation.
Short- And Long-Term Ways To Drive AI Cost Efficiency
1. Short-term: Allocation and attribution — track what’s driving your AI spending and who’s responsible for it
If you’re making a big bet on AI, you’ve probably got centralized research teams building crazy, complex, interesting models on top of all your company’s data. As a result, AI costs tend to be large, amorphous blobs of spend that are difficult to trace back to the correct features, products, teams, microservices, etc.
What you should do today: Put systems in place to allocate your costs to the appropriate sources. You likely won’t face pressure for a while to turn these costs into profitable investment centers. But that pressure will come, and when it does, you won’t want to scramble to organize a rat’s nest of spending. Allocation is a project best tackled proactively.
Part and parcel with allocation is attribution — determining which teams/engineers are driving which portions of your spending. It’s true in the cloud, it’s true in AI: Every engineering decision is a buying decision, and the only way for your engineers to make responsible buying decisions is with real-time data on what their infrastructure costs.
Allocation and attribution are CloudZero’s superpowers. Regardless of how good (or not so good) your tags are, CloudZero CostFormation® can programmatically allocate 100% of your AI costs in a matter of hours. Proactive visibility is a shield against overspending, and no one does it better than CloudZero.
Learn how complete cost visibility lets Hiya manage cost growth and project the cost of ventures in new geographic regions.
2. Long-term: Track your GenAI Efficiency Rate
The day will come (if it’s not already here) when your exploratory AI projects turn into revenue-generating products and features. The day will also come when those products and features are expected to be profitable — and as anyone who’s tried to get funding in the last couple years knows, investors really like profitability.
You’ll eventually want to take a more unit economic approach to analyzing your GenAI costs. Since, by this point, you’ll have accurately allocated your costs to the right customers, features, products, teams, and/or microservices, you’ll now be able to compare each unit’s revenue to its costs and derive a GenAI Efficiency Rate — overall, and within specific units.
Like the Cloud Efficiency Rate (CER), the GenAI Efficiency Rate simply compares GenAI-based costs to GenAI-based revenue. It sounds like a simple metric, but you can only get an accurate view of it with a foundation of excellent allocation. Once you know which dimensions are most and least profitable, you can make informed decisions about where to drive the most impactful efficiencies.
Learn how CloudZero improved HighRadius’s CER by 5%, getting it into elite SaaS territory (90%+).
Lay A Foundation For Proactive GenAI Efficiency
Again, it will probably be a while before your AI projects face pressure to turn a profit. But the time will come, and when it does, you’ll be a lot better off if you’ve got the pieces in place to track and manage AI costs. Allocation, attribution, and unit economics are CloudZero’s superpowers — and we’d be more than happy to demo them for you.
The Cloud Cost Playbook
The step-by-step guide to cost maturity
