▲ 594 points
back
312 comments
I have updated OpenAI's chart[1] from yesterday to include one more datapoint: DeepSeek V4 Flash 0731. It's on the frontier.
https://files.parasmittal.com/openai_aa_luna_dsflash.svg
1: https://openai.com/index/advancing-the-price-performance-fro...
LOL https://artificialanalysis.ai/models/deepseek-v4-flash#intel...
High hopes for V4 Pro
Haha, and I almost felt bad after seeing this chart yesterday.
> For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework
So, are they planning to announce an optimized coding agent harness as well ? DSv4 flash is a fantastic model, and my daily driver. With reasonix or pi, I can code all day long and pay a few pennies for it. No token anxiety. Whereas the same model with fireworks/openrouter, with zdr thrown in, token costs ratchet up with no explanation. Likely that the model is subsidized for gathering usage data. I am waiting for the day I can run this locally.
They announced it already, read the tech report.
"For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the max reasoning effort level with temperature = 1.0, top_p = 0.95."
If you're not paying for the model through Open router, then which provider are you buying it from? I thought that if that provider was listed on open router it would be the same price as going directly to the provider? Are you buying from a provider that isn't listed on open router already?
Somewhat relatedly, how do the economics for Huggingface work? They must be hosting petabytes of models and datasets by now. I have downloaded quite a few “just in case”, only to replace them with the later iteration months later.
Does the file hosting actually cost peanuts when you do it yourself and the cloud has shattered my understanding of what it actually costs to deliver so much data?
File hosting is pretty cheap. Egress traffic is about $90/TB in the cloud, but around $1/TB in the real world. Storage is in the realm of $5/TB/month after adding redundancy
At the scale of Huggingface, that still amounts to a lot load of money. Significantly less than if you did the same in AWS, but still a lot
That said, they do have a deal with AWS to make the data available in AWS ip space. Maybe they got some cheap hosting out of that too
HF uses chunk-level deduplication that greatly reduces the amount of physical storage needed for a given amount of logical storage.
Nowadays storage and bandwidth is cheap as long as you're not going through a hyperscaler and it doesn't require compute (ie. "dead" data). I'm working mostly on GCP so I'm not sure what are the rates now, but at least AWS were known for billing insane numbers on egress.
As long as you're smart on caching on the edges and deduplication/overlaying with your content (which structured data types like models, containers and repos typically are fit for) you can get remarkably far for less than you'd think.
The economics of serving models seem to be way more dodgy.
Bandwidth is really cheap when you run your own infra
Already beat Luna on price/task, by about 2x:
https://artificialanalysis.ai/models/deepseek-v4-flash?intel...
If deepseek v4 flash is beating DeepSeek V4 Pro, can we expect new V4 Pro which is on par with Opus 5 in couple weeks (even better if it beats Opus)?
The really interesting thing about this is how big of a jump was achieved with just extra fine-tuning here. No structural changes to the model, just more data, compute and time. It makes me pretty excited for the future of small models - DS v4 flash is a relatively small model when compared to the class it's competing with, so likely similar gains can be made applying quality data/training pipeline to other smaller models.
So GLM 5.2/Gemini 3.6 level intelligence for $0.28/m output. And their updated Pro model coming soon....
Plus a size you can genuinely run at home: Unsloth lossless Q8 at 162GB.
This seems to me like this is probably at least a large part of what OpenAI was up to yesterday with their aggressive price cutting; trying to get out in front of this.
If the full non-flash model follows up with the expected improvements, and at the price point they've been keeping, it puts the frontier labs in a tough position and it feels to me like like OpenAI is reaching deep into their pockets to try to head that off.
TFA link is a 404 though. I'm reading through https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 instead
It’s exciting that a model scoring this high is dirt cheap.
It’s also so inefficient, when they release the full performance numbers it’s not going to be good.
One example, it takes about 3.6x more tokens to finish the same work as Gemini Flash 3.6.
For people with single RTX PRO 6000 96GB or DGX Spark 128GB, vllm-moet is a very good engine, although lesser known. It auto generate a symmetric 2-bit plane for inference and also generate a 4-bit delta cache to recover precision. Support ssd streaming oversized weight. You pick how much VRAM to allocate to each to balance out speed vs precision. 170 tps with ds-v4-flash demonstrated.
It use the stock model, no new models requires.
Worth spend a few hours to try.
The DGX Spark requires a small hack to ignore the difference between sm120 vs sm121, but it does run on sm121.
The weights were just released a few minutes ago: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
If it matches GPT-5.4 on coding tasks (as benchmarks suggest) this could be my forever model. And with partial SSD streaming, I could run it locally today. :D
New Deepseek models are like Christmas for me. Really big fan of low cost API models, noone does it better than DS. Until VRAM price is low enough to run models locally, this is the way to go.
The subsidized subscription model won't last, API pricing "feels" closer to a true sustainable business model.
Here is their blog article: https://artificialanalysis.ai/articles/deepseek-v4-flash-073...
What's most impressive here is that that DeepSeek keeps showing how much of a performance improvement can come from post-training alone while the architecture stayed the same. It's a strong reminder that we're probably underestimating how much optimization is still left after pretraining
I have been loving deepseek flash. I can code in my hitl preference doing complex but moderate amount of coding and not even hit 1 dollar in a session. It gets harder to justify paying 20/mo to Anthropic for unused compute when I have it on demand and at a much cheaper rate with deepseek.
The model is already up on Opencode, but they require a consent to use Chinese datacenters.
I was writing a benchmark for my own harness, and DS4 flash answers as well as Fable 5 on any query.
The specific agent is focused on getting precise and on point answers about a codebase.
The starting point was nowhere near. E.g. asked why was X implemented in a certain way it would give bogus answers when the real answer was that there was no reason at all.
The benchmark included more than 50 questions or different difficulty.
But when the agent was improved in its prompt and rooting it was impossible to have it perform worse than closed source sota.
Just to say that the quality of the harness is as important as agents intelligence.
I got a disappointing pelican on default reasoning mode: https://static.simonwillison.net/static/2026/deepseek-flash-...
But a REALLY good pelican on reasoning mode high (via OpenRouter): https://static.simonwillison.net/static/2026/deepseek-flash-...
If any editors are reading this thread, the weights are on HF now, but the 'open source' question still implies it's proprietary/API-only.
Performance per dollar is in the ‘too good to be true’ territory, what’s the deal here?
Wow this thing just blew out the pareto front for intelligence/$
Is the "Output Tokens per Intelligence Index Task" data actually correct or am I reading it wrong? It says there that "Kimi K3 (Max)" would think/reason less than than deepseek-v4-flash, and a whole bunch of other models, like less than hy3 and even gpt-oss-120b, but in my experience, K3 is probably the model that thinks/reasons the longest of all of these.
Am I just using it on tasks that makes it go on forever vs these benchmarks that are short&sweet, or something like that? I've been throwing bunch of identical prompts at different models at the same time, and when comparing hy3 and K3 I've never once had K3 reason less than hy3, as just one anecdotal data point.
I’m wondering if they did anything to address the DSML tool calls leaking. Has been an issue with both Flash and Pro so far.
Unfortunately, DeepSeek Flash still doesn’t support multimodal; otherwise, it would offer better value than GPT 5.6 SOL.
Is artificial analysis using the 80% reduction in Luna pricing that was announced yesterday in these charts?
> DeepSeek V4 Flash 0731 (Reasoning, Max Effort) is amongst the leading models in intelligence and well priced when comparing to other models of similar price.
Similar price? Doesn't make sense. Maybe they meant power, capability or speed?
I hope they somewhat fixed the hallucination and forgetting plagued V4 previews and that it wasn't just benchmaxxed but the numbers hold in reality. Then it would be my choice for 2x DGX Spark or 2x RTX Pro 6000.
If anyone from ArtificialAnalysis is here: The Intelligence card on the page seems to incorrect. It shows different data (#2 for Deepseek) than the actual Intelligence chart lower in the same page.
Really want to use this, but for web page design you really need vision!
Here's you pelican just in case you were guessing
404 ? It's very strange that this can't be fixed.
I can't help but feel the timing coinciding with luna's price updates to be somewhat strategic. But without multi-modality it's slightly dead-in-the-water for my usecase: https://design.withfudge.com. I'm currently using Minimax-M3, but Luna ekes out abit futher on the intelligence, so i'll be switching to it very soon.
Would be awesome to see a new ds4 release. Having so much in something that can be run locally is mind blowing
No speed (tokens/s) benchmarks?
Have not had great ds performance in agentic harnesses in the past compared to glm or k3
here it is on openrouter https://openrouter.ai/deepseek/deepseek-v4-flash-0731
My problem with DS flash/pro is that they don’t push back on obvious bullshit, both irl and code [0] but it’s a great implementer workhorse if you give it _very_ detailed specs.
[0] https://petergpt.github.io/bullshit-benchmark/viewer/index.v...