The Era Of Efficient Engineering: 3 Takeaways From AWS re:Invent 2023

I GOT MENTIONED BY AMAZON’S CTO!!!!!

Sorry. Just had to get that out of the way.

Another year, another pilgrimage to Sin City to rub shoulders with the architects of Amazon, to talk shop with the world’s leading software companies, and to hear the latest announcements from all the wave-makers on the expo floor.

It’s also an opportunity for the cloud industry, now endowed with the very serious task of keeping much of our economy humming, to live a little. As you can imagine, Las Vegas and its parties never fail to … let’s just say “impress.”

re:Invent is always a whirlwind for me, this year — a year in which CloudZero raised a $32M Series B in the weakest funding environment in years and grew the company over 2x — more than ever.

I spent my time presenting with Kyle Daigle, GitHub’s COO, about our use of AI during a breakout session and had amazing conversations with many of our customers during dinners, meetings, and filming at Blue Wire studios with Myer Media (actual podcast footage to be available later.)

On top of all that, like an early Christmas present that fell out of Santa’s sleigh as he made that last-minute trip to the blackjack tables, Werner Vogels — Amazon’s aforementioned CTO — put cloud cost front and center in his keynote address, then published “The Frugal Architect” moments later. 

For those wondering, the last time cost played nearly this prominent a role in a Vogels keynote was 10 years ago — 10 long, rollercoaster-esque years, where he mentioned cloud cost as a good thing during his re:Invent keynote back in 2013.

He actually said “Nobody’s going to say 10 years from now, ‘I wish AWS was a bit more expensive.'”! I was there then, and here we are now, a decade later, with absolutely no software company asking for AWS to raise their prices.

And talking about cost wasn’t just a fluke that year either, he discussed cost at the very first re:Invent in 2012, where he said everyone should “Architect with cost in mind.” and shared an internal Amazon slide from 2007 where he highlighted this as well.

Werner was so very right back then. I truly believe down in my soul Werner has always seen cost efficiency as one of the clouds superpowers, but AWS as a whole seemed to have forgotten about this key insight. So my mind was blown this year, when in 2023 he finally at long last, returned to talking about cost.

My thumbs almost tripped over themselves as I tweeted (or X’d? I don’t know) at Vogels during his address. I then proceeded to fall out of my chair when Vogels came back onstage and quoted my tweet — his exact words: “I think someone wrote on Twitter, ‘Every engineering decision is a buying decision.’ Keep that in mind.”

Werner has spoken: I’m now in the process of changing my name to “Someone on Twitter.”

I mention this not to stroke my own ego (it’s big enough, trust me). I mention this as a way of segueing into my key takeaways from re:Invent 2023 — which, yes, have a lot to do with efficient innovation in the cloud.

Years ago, I remarked that “Every engineering decision is a buying decision,” and that cost must be a key engineering metric when designing cloud systems. Last week, when Werner Vogels picked up the mantle, and doubled down on cost as a non-functional requirement he (along with Nubank, a CloudZero customer) validated the mission CloudZero has been on since our founding in 2016.

Building well-architected cloud software requires engineers to consider cost as part of the design.

3 Key Takeaways From AWS re:Invent 2023

1. Companies are spending money on AI like it’s the cloud in 2009

No one in or out of the software world could survive 2023 without having the term “AI” rattle around in their brains, rent-free. So, going into re:Invent, I was expecting to hear a lot about AI — and re:Invent did not disappoint, it was mentioned by some counts over 300 times … just not the way I expected.

Not only are companies spending a lot on AI, they’re proud to talk about how much they’re spending on AI. I had numerous conversations that sounded something like: “We’re throwing money at AI. Isn’t that awesome?”

I’m as excited about AI as the next guy (probably more), but in a year that was as hard on software companies as 2023 was, it surprised me at first to hear people talk about spending on anything this way.

Of course: Sophisticated AI is still new enough that people are excited about it, so excited in fact that they really aren’t worried about cost. But the true reason the music hasn’t stopped is because your business and your CFO doesn’t care yet. To explain this, I have the following theory: Just like the early days of the cloud, most AI spending is still being allocated to R&D — meaning it’s not a problem.

Yet.

I remembered that not all dollars spent on the cloud are equal — something that’s often hard for engineers to wrap their minds around. I experienced this first hand with some of my early experiments with AWS, back in the late 2000s and early 2010s. Building in the cloud was all new work at the time, meaning the costs of cloud engineering were chalked up to R&D. 

This is exactly where all of the AI spending is getting allocated right now: to an R&D tax write-off — giving everyone a (near-)carte-blanche playground in which to try and err.

If we’ve learned anything about cloud spending in this economic downturn, it’s that an out-of-control cost center can sink you when macroeconomic tides turn, but where you allocate that cost matters, particularly when you think about SaaS metrics. AI spending will eventually migrate over to OpEx and be counted as COGS, and those AI budgets won’t be the kind of no-limits hold ’em that they seem to be right now.

When that time comes, and AI costs start counting against product margins, all hell is going to break loose. My recommendation to you now is to get ahead of this apocalypse by making sure you can connect every penny of your AI spending to your engineering teams and the features they are building.

What’s the actual cost of that cool new GenAI chat interface you just rushed to market? Wait, $17 dollars per question you say? We are so going out of business…again.

Of course, CloudZero solves this, unit economics and allocation is our superpower, just saying.

2. The market is singing a tune called “Efficiency”

I’ll get back to Vogels and Amazon in a moment (I’m as excited as you are). But before I do, I want to single out a nuance in the way that companies were talking about saving money in the cloud.

I was expecting to hear companies talk about how much they’re spending in the cloud, and how important it is for them to get their costs under control. I also imagined that I’d hear more of the same old way of thinking — that reducing waste, rightsizing and committed use discounts can get you all the way there.

And, well, I definitely did hear a lot of that. A lot of people are still mired in the idea that cloud costs are a problem for finance teams alone, accompanied by the cynical idea that engineers will never be willing to truly engage with cloud cost issues.

But I was heartened by another aspect of these conversations: that many people were talking about “efficiency,” not just “savings.” This point was made loud and clear by Nubank’s Cat Swetel, senior director of engineering, who was on the mainstage with Werner discussing how they balanced cost with stability.

Nubank is a CloudZero customer, and it’s been heartening to see how focused they are on building cost-efficient software in AWS and they have placed their engineering teams at the center of those efforts. We are truly honored to have them as a customer because they are leading, and as Cat demonstrated, it’s been wildly successful for them.

This is more to my taste. Thinking about cloud cost purely in terms of spending less is a reactive, shortsighted way to think about it — especially as we enter the age of AI. Thinking about it in terms of efficiency — getting as much out of every cloud dollar you spend as possible — is the way to build sustainable digital businesses.

Framing the cloud cost crisis — that is to say, opportunity — in terms of efficiency is a great first step, and uprooting the cynical idea that engineers are incorrigibly disengaged is key.

If you need some inspiration, look to our friend Dr. Vogels.

3. AWS gets it, and we get AWS

So you get it, I could scarcely have been more excited about Werner Vogels’s keynote address. I’m not saying Amazon is finally listening to the experts at CloudZero, but did I mention he did repeat my catchphrase: “Every engineering decision is a buying decision.”?

A couple of the things he said that really stuck out to me:

One Small Step For Engineers; One Giant Leap For Engineeringkind

Here are the seven laws of Vogels’s “The Frugal Architect”:

It’s the seven-beat rhythm that CloudZero has been drumming for years. It’s an anthem that says, “Make cost-efficiency a priority from day one, and know how to manage it over time.” It’s coming from a company whose greatest innovation might not be the cloud or e-commerce, but their business model — their radical philosophy of building efficiently so that they can offer their customers an outrageously competitive price point.

I hope Vogels’s comments this year signal not just lip service in a tough economic climate, but the first step toward a fundamental change in the way software is developed.

If you feel the same, there’s no better partner than CloudZero in making it happen. Get complete visibility into 100% of your cloud costs. Empower your engineers to build cost-efficient software — without slowing down innovation. Power elite cloud efficiency.

One last thing …

I GOT MENTIONED BY AMAZON’S CTO!!!!!