▲ 293 points
back
341 comments
I remember at Google at around 2007 - 2009, as Google was massively expanding its data centers, there was a lot of unused capacity, especially during off-hours. Any engineer could run as many jobs as they wanted at zero priority, which means the job would be first in line to be killed if a more important task needed the resource.
I did so many interesting experiments with MapReduces that would run overnight.
For a while, I would even build internal services that were basically "free" because I'd just run them all at priority 0.
Over time those services got less and less reliable as overall usage started to increase, so I was forced to either justify the resources or scale back - but that was a good thing.
I feel like something similar would be a good model for AI token use: big tech companies ought to have their own self-hosted LLM data centers to power their own needs, then let employees use off-hours capacity to experiment.
Outside of experimentation, we should be encouraging token efficiency for everyday tasks. Rather than having a certain number of tokens, engineers should be evaluated based on how much they actually get done.
Using a lot of tokens to automate a process that used to require hours of human labor every week? Good use of tokens, should be encouraged.
Using a lot of tokens to debug an easy frontend bug that could have been fixed by hand, and still took you 4 hours to complete? Waste of tokens, should be discouraged.
Isn’t this like batch processing offered by oai? Your request gets done within 24 hours at half off
"Using a lot of tokens to debug an easy frontend bug that could have been fixed by hand, and still took you 4 hours to complete? Waste of tokens, should be discouraged."
Hahahah good luck with that!
For many of us, what is happening now was super obvious. Telling a new formed crack addict (who you wanted to become addicted) to be more thoughtful about their consumption of crack... yeah not gonna work is it.
Don’t think we’ll see similarly logically behaviour from LLM users tbh. A sizeable portion of the user base seems to insist on through opus at every trivial task
Most AI front ends seem to be designed for interactive jobs, so they make it hard to define a job that should be done eventually with zero priority. It makes much more sense to do that with spec-driven development (have work done with the human on the loop rather in the loop), but as far as I know that just isn’t well supported by any front end yet (would be happy to be proven wrong, my experience is with Google front ends).
You’re right of course, but wouldn’t it be more likely that everyone will embrace 10x cheaper Chinese models?
If any company announces that they use token consumption as an employee performance signal, for me that's close to a red flag to stay away from that company.
No company with good engineering leadership should act like this is remotely a good idea.
As I have snarkily observed at work: if I go $100 over the meal allowance on my business trip, I'll have to have an unpleasant conversation with my manager or finance. If I use $500 in AI tokens unproductively I'll be recognized for being a top AI adopter.
You'd be surprised, I know a few devs in very big tech companies, not faang but you definitely know them, and they all have some kind of token leaderboards, a few told their dev "we don't want you to write a single line of code manually anymore", etc.
I assume the execs perspective is something like: if the top 20% of worker produce 80% of the code with LLMs and the company still works then we can get rid of the bottom 80% of devs and save money
Tokens are the new "lines of code per engineer". Easy to graph, easy to "manage".
Meta does this. Guess what one of the criteria for their recent layoffs was.
There is little new under the big fusion reactor in the sky. I just read a chapter in James Glieck's "The Information" about tokenmaxxing in the telegraphy industry. There used to be a big market for code books to reduce the per-character charges for sending telegrams. Compression was cash in the pocket. The telegraph companies discouraged the practice but were forced to accept it. The telegraph code industry started with the initial commercialization of telegraphy and didn't end until the 1920s.
There was a cost to it though. Codes greatly reduced redundancy, and caused large miscommunications from very small errors. As Glieck explains it, this was the opposite of the African drumming practice of adding redundancy to strengthen the relationship between the rhythm and the language that the drums mimic.
I always used to wonder this about software stacks even prior to LLMs, but it seems more relevant now somehow:
When will Uber (or your favourite company) be 'done'? They've been writing software for 16 years.
They match drivers to passengers. More software isn't going to increase the chance that I seek them out instead of taking a bus or train.
Will their software be finished in 20 years? 80?
Tokenmaxxing makes no sense, it is akin to write extremely inefficient SQL / Spark Jobs, full of cartesian joins, ultra skewed datasets, etc, just for the sake of using as much compute / memory / IO as possible.
This always happens when the metric becomes the goal, companies should nurture and foster an environment where AI is used in the most efficient way possible, first asking "do we really need an agent for this" and if so, what kind of agent is needed, what model, reasoning level, etc.
They should also promote projects that aim at saving tokens, increasing cache hits, codifying the information in ways such they use as less context as possible (graphs of knowledge are pretty good for this!)
I don't like using AI. I don't find it particularly helpful. But my employer insists that we use it and tracks metrics so I make sure to give it pointless busywork daily. That way I show as using it even if it causes more problems than it fixes.
I am certain that the max sustainable boost from AI use -- with code review and otherwise all-in -- is approximately 20% with the appropriately skilled senior engineering talent, and the token budget for any engineer should not exceed that.
I do not believe that engineers who are tokenmaxxing are truely productive and I have not seen any evidence whatsoever (perhaps the opposite).
I've personally found that with the right flow and codebase knowledge, that's achievable with sustainable levels of effort.
AI for engineering productivity seems to be widely misunderstood to be a magic button that produces the same result, but faster and more cheaply. And based on that reasoning, you should want to force employees to tokenmax, because, why wouldn't you want to get more results but faster and cheaper?
A more nuanced view would be something like:
* AI lets you achieve your roadmap somewhat faster, but:
* You incur tech debt that's similar to if you hired a dev temporarily for the features. You don't necessarily have someone on the team that understands the new code.
* Similarly, you aren't upskilling your junior team members. So you aren't getting skill/wage arbitrage as much as before.
* You will complicate the product. P2 features are P2 for a reason, but AI can cause them to be included and complicate the product for lower marginal gain.They are burning money to pay for AI-assisted development. Ok. But what is the ROI of it all? Was it worth the supposed increase on efficiency?
Why nobody talks about those points, which are actually the only interesting points of all this AI craze?
Maybe don't use the most expensive models on the planet? Maybe use AI like a tool and not this black box that grants wishes?
I'd be interested to know if this is about individual employee AI usage, or use of AI tokens in production features, or both - and assuming both, what the split is.
I can see how Uber could burn unbelievable amounts of tokens if they start running internal features that run a bunch of prompts against every completed ride, or every customer profile, for example.
Or maybe this is about employee usage, but they introduced some stupid "you get evaluated on how many tokens you used" thing a couple of months ago when that was trendy and are just beginning to notice how much that cost?
I find it shocking that anyone ever thought tokenmaxxing was a good idea.
AI maximalists like to compare the technology to electricity. Imagine if in the early days of electrification, a CEO had rewarded staff for increasing the amount of electricity they consumed rather than finding ways to use it for business impact. Institutionalizing people who showed signs of mental illness was popular in those days, and I suspect that would have been the outcome.
It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way."
Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits, to get to the top of the AWS leaderboard. If you don't do this, you're ranked at the bottom, and good luck at the next review cycle. Who could have expected that this is not the road to success?
Waiting for tokenedging next.
Limits are beneficial. They should be treated as a design feature, not just a stopgap.
When something is abundant, people tend to waste it.
I’m perfectly happy with my base subscriptions. I have Claude Code and Codex monthly subs, plus a yearly Google AI Pro account because it was a logical upgrade from the cloud storage plan I already had. I think it worked out to something like an extra $10/month for the AI features.
I constantly rotate between them during the week, managing tokens carefully, cleaning sessions and contexts as soon as possible, and being intentional about usage.
I honestly don’t understand the appeal of these ultra-expensive max subscriptions.
It reminds me of that flying orb toy I bought for the kids a few years ago. The battery only lasted about 10 minutes, and the kids would go ape shit crazy while it worked. Then it needed a 30-minute recharge, which created a natural cooldown period.
I actually considered that a good feature. I would never want the thing running nonstop.
"He said that, based on talks with Uber's senior engineering leaders, he realized higher token usage did not translate into a proportional increase in useful consumer features."
He's saying that like it's some grand epiphany and not the most self-evident, obvious thing I've heard this month. Some of the literal dumbest people on earth are in charge of these major companies.
Protip: skunkworks type side projects are a great way to do tokenmaxxing when you don’t have enough work coming in, but still need to burn tokens to look productive. And because side projects are only governed by you, you can truly go nuts and let scope creep run wild. Soon enough, you’ll be one of those engineers burning six figures a month on AI and people will be in awe of your abilities, probably even elevating you to key AI evangelist positions within your company. And if you actually create something cool, you’ll be praised for your use of AI, and you can just say you built it all in a day or two instead of slacking off for months on your real work.
As with many things, users will discover a happy medium. There is scope for a lot of productivity gain here if the C-suite is willing to understand the tech and work with engineers rather than whatever Dario Amodei is selling.
I have Opus 4.7 at work at 15x. Burns through tokens like water. It feels like one of these new mega datacenters is just for me. I'd love to know what the bill is, but we're just encouraged to do as much AI as possible.
I actually do think token maxing is good, but they should have limited it per user. I find it reallly hard to get people to max out the Claude $100 plan, let alone the $200 plan. I understand the enterprise plans are different and more expensive, which is how you get these kinds of issues. But encouraging people to try things with AI is very important, and some amount of token maxing is importsnt.
AI productivity hasn't been well studied yet, but I'm betting that we'll end up with some variation on Price's Law, I.E. some small subset of workers get most of the benefit, while most just burn tokens with little to show for it.
I also want to call out the false productivity opportunities AI offers. There are whole teams building their own "gas town" and not shipping features.
Clearly they need more layoffs, and for that matter why keep anyone around? After all, AI will be writing 100% of code in 2026.
Not all tokens are created equal. It's easy to use a ton of tokens by having agents work together in parallel. That's basically the equivalent as people spending time in meetings, hardly a productivity win. As with everything in development, results matter, how you get there doesn't (unless you're a bad manager).
tokenmaxxing is becoming harder to justify could be a change in the labor market => when capital was free the companies optimized aggressively around retention and internal status spending but high rates + slow growth oblige firms to back toward productivity and operating leverage.
On token consumption and efficiency... AI-champion guy in my prev company made a metric, like how many tokens are spend per line of generated code, and even put a leaderboard based on that metric, praising guys with the cheapest LOC.
For me that's insanity for so many reasons...
Replace Tokens with Gas, or water or healthcare or anything - and it's foolish. You shouldn't let the seller dictate what amount you need of something.
Smart engineers are figuring out how to best use their tokens - as tokenmaxing is just as silly as gasmaxing your car.
Oof leader of bubble are starting to take a step back?
The black bill that is coming that nobody is prepared for is that the value of a token varies greatly depending on the human. Companies will quickly find out its much better to give your top 10% engineers a lot more tokens and lay off your average engineers. The 10x engineer will become the 1000x engineer.
Wrote about this and the impact of to jobs here: https://x.com/deepwhitman/status/2058324179506831372
I just realized my company is months behind this curve. About to blow my token allocation. Before I do, anyone have requests? Sincerely.
Feels like they are debating internally whether to cut people or AI spending. Very healthy debate. Let's hope they spare people.
I’m genuinely curious why they don’t cap at $100/month Claude Max per employee. That would be sufficient for 80% of them.
At what point is there a difference between a burn rate and tokenmaxxing? Isn't it the same as during the dotcom bubble?
many of these leading AI companies are operating at large losses and subsidizing users with VC money. Profitability will entail having to impose greater limits and raising prices, so this will reduce to some degree the value proposition of AI compared to humans.
Why do keep doing this? It's the same as measuring by LoC, we know it's not gonna work. Also, see Goodhart's Law[1]
Levie’s Law of AI Psychosis
>"He said that, based on talks with Uber's senior engineering leaders, he realized higher token usage did not translate into a proportional increase in useful consumer features."
Goodhart's law strikes again at someone with enough power to be both ignorant of it and make others suffer their ignorance. You cannot simply measure productivity by tokens spent just like you can't measure it by hours spent in a chair at a desk.
As soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.
Not the first time supposed leaders ran into Goodhart's law.
At what point might it be cheaper to, say, hire a human?