back
89 comments
Interesting to see that an RTX 3080 is faster than a dual RTX 2080, and more than twice as fast a single RTX 2080, which is consistent with one of NVIDIA's claims.
RTX 2080 - 10 TFLOPS, 448 GB/s

RTX 3080 - ~30 TFLOPS, 760 GB/s

So for most things compute you'd expect anywhere from 70 % to 200 % performance increase. Note the significant increase in operational intensity in this generation, to about 160 FLOps per Float Load/Store (up from ~90). The fact that we're not seeing 200 % increase more widely can point at things like this being a problem for existing applications (so RTX 3080 is _even more_ memory constrained as previous cards [1]), and perhaps also some applications struggling feeding enough work items / scheduling issues in general.

[1] Alternative view: Even more operations you can do for free when you have to process a given buffer anyway!

>~30 TFLOPS

Jesus.

Also 3080 is much cheaper than 2080 was at its launch.
Same or similar price as 2080, cheaper than 2080 Ti.
For a comprehensive benchmark of various deep learning operations on different GPU cards, see http://ai-benchmark.com/ranking_deeplearning.html. You can run it also on your computer! (I did, just to see if all drivers are installed properly and my results match the other users.)
I know these are just preliminary but I am really excited to get my hands on a few of those cards. This is absolutely amazing! Thanks for sharing the benchmark.
The fact that the not even top of the line (3080) card is knocking the door on 8K gaming is amazing to me.

It's still not cheap but who says moore's law is dead (I know its apples and oranges, but 60% gen-to-gen performance is great). The first games I played were already 3D and looked ok, but the idea that we'll probably be playing movie-quality raytracing within a decade is really something to look forward to.

What's really freaky is that the RTX 30xx series of cards aren't even being manufactured on the current-gen process!

NVIDIA is using Samsung's 8nm process, which has about 60 million transistors per square millimetre (MTr/mm^2).

That's not cutting edge! The crown is currently held by TSCM's 5nm process, at 173 MTr/mm^2.

Some time next year, TSMC is starting "risk production" of their 3 nm process, which is expected to hit about 300 MTr/mm^2. That's a solid FIVE TIMES higher density than the process used for the RTX 30xx series.

Unlike general-purpose CPUs, where transistor density does not linearly translate to real-world performance, GPUs are designed for embarrassingly parallel problems and have nearly linear scaling. More transistors equals more "CUDA cores" equals more performance.

The only thing holding back GPU performance is memory bandwidth. Current-gen consumer cards are just shy of 1 TB/s of memory bandwidth, but to get 5x performance, they would need 5 TB/s memory throughput to match. That's... difficult. Even with HBM2E, you'd need to stack a bunch of them to get near that.

But yeah. 8K gaming is crazy. Real time raytracing was an utter fantasy just a few years ago, and I just played through Control at 60fps and it was a visual feast.

I grew up in an era where wire frame 3D graphics took seconds to redraw the screen. I used keyboard macros to control a CAD program because it had no hope of keeping up with mouse movements.

My unborn son is going to grow up to play in a world of 8K raytracing as standard, with visuals better than Pixar movies of just a few years ago. That blows my mind.

> Real time raytracing was an utter fantasy just a few years ago

We still don't have real time ray-tracing. Even the demos that only use ray-tracing are throwing the strict minimum of rays and they apply a series of complex filters (using machine learning) to remove the artifacts and the noise.

The challenge is heat (and cost) for the latest gen. GPUs are already prone to heat challenges, I can see the latest process requiring liquid cooling with their current design(s).
> What's really freaky is that the RTX 30xx series of cards aren't even being manufactured on the current-gen process!

Exactly. This is why I'm not too worried about Intel not being on the smallest current-gen process, even though this AMD fans are jumping up and down about this.

Something noteworthy is the higher nm tend to be a bit faster than the smaller nm, so Samsung's 8nm isn't going to be slower than TSMC's 7nm.

The reason lower nm is better, beyond cost savings, is heat output and power consumption. If heat becomes a problem, the chip has to limit itself, as we're seeing with laptops. Nvidia's new cooling solution seems to fit the bill just fine, so 8nm is no problem.

I suspect the next generation is going to be on 7nm, is going to be a bit faster and will consume a fair bit less power, which will be nice, especially if you plan on training neural nets all day.

Since Radeon VII's HBM provides 1TBs of bandwidth with a 4096 bus just up it to 20,480 bits and bam 5TBs.

Not sure how you'd fit 20 stacks on an interposer. I can't solve all your problems.

> The only thing holding back GPU performance is memory bandwidth.

Cooling and getting enough power to the chips is going to very challenging. The 3080 already needed a redesign of its cooling solution and power delivery mechanism. At 5nm or 3nm, things are going to be a lot more difficult.

Moore's law is about transistors. It's not dead, it's alive.

The problem is you can't use that law to increase speed of CPUs. You don't need more transistors, you need faster transistors and Moore's law does not help with that. We had 4GHz 15 years ago, most of CPUs still work under 4 GHz nowadays. But with more transistors you can implement some common operations in hardware (like crypto, vector operations). Also you can just increase core count. Or you can put energy-efficient core along with energy-hungry core. And those things happen with CPUs. Unfortunately many workloads are still single-thread capped.

GPU on the other side is inherently multi-threaded. You need 8k? You just need 4x transistors compared to 4k resolution. So GPUs will evolve even further and there's no limit, at least until we hit transistor wall.

You have to compare performance per watt though to see the real gains. They can't keep pumping more and more watts through these things indefinitely, surely we are already at the limit.
This is one thing that impressed me with the 3080 .. it looks like 2 of these will be around the performance (for some compute tasks) of 4 x 2080Ti The 4 2080Ti would max out at 1KW the 2 x 3080's at ~700W ...
Well I can supply plenty of watts, but at some point I reach the limit of performance.
The 3090 has a TDP of 350W which is a lot but it's not unprecedented either. There have definitely been cards from even the Fermi days in that region (590?)
Keep in mind that video games are optimizations on top of hacks on top of optimizations on top of hacks on... This is necessary, because we simply don't have the number-crunching capability to simulate the video game world better. If developers could have that horsepower, then they'd probably use it. In other words: as people get faster hardware video games will start using more computational power. This allows developers to scale back some optimizations and usually allows some different styles of games.

Another thing to consider is that the number of pixels pushed to your screen is one thing. Another thing is that you will need higher quality assets with more detail too.

> It's still not cheap

Its funny to me that some people complain about nvidia price gouging etc, but if the sales of 3080 are any indication then based on basic supply&demand idea if anything they are massively underpriced. Lowering the prices would seem to just mean that more people would be disappointed from not actually being able to get them because the supply is so limited. The little I know about chip manufacturing also suggests that its likely that fabs have been producing ga102s at full capacity and its not like you can just get another fab to produce more of them in blink of an eye, so availability is probably really just constrained by production.

I think this is a marketing problem, where selling the 3080 at $1200 would hurt future sales once supply meets demand, even if the price is lowered later. Maybe the optimizing solution would be something (awful) like a car dealership where the price is negotiable based on volatile local factors.

As a complete layperson when it comes to economics, I’m curious if someone has extensively analyzed ‘market value’ vs MSRP over the lifecycle of a product rollout and come up with a workable formula for calculating the optimal price.

8k gaming? Supposedly 3090 is capable, but I gotta ask - who games in 8k? 8k monitors are still far too experimental.

Not to mention that scaling is going to be a pain.

The Endgame has to be VR at this point - 4k is more than enough for "non-gameplay" games (i.e. I play CSGO at low settings in a stretched resolution but I play Assassin's Creed as hard as she'll go cap'n).

Until the angular resolution of a pixel is good enough an inch from your eye there will be increasingly higher resolutions in the HMDs

Prepare to wait a bit if you didn't secure one yesterday. The 3000 serie launch is the exact definition of paper launch.
Apparently the supply of hardware was similar to the last few nvidia launches and it was demand which was substantially higher this time.
paper launch means no hardware. This is just very limited supply. The only atrocious thing is the FE edition is 100$ less then the cheap AIB models.
It's just scalpers ran swarms of Selenium against every web storefronts. Happens for everything buzzes on the Internet these days.
Do the resnet results use the new sparsity feature ? I'd be interested to know what impact does that have.
What is this feature and can it work in generic matrices? It could be a game changer in physics and operations research where the matrices are sparse.
I don’t think “sparsity” in ML, where like 10...50% is sparse, is the same as sparsity in Physics, where 99.999% (add more 9 with larger problems) of your matrix is sparse.
Half the entries in your matrices need to be 0, then the Hardware will compress them and execute the matrix-matrix multiplication 2x as fast. On a tensor instruction level.
It should, AFAIK that was on the SM-level.

> Ampere's benefit is that it can deal with dense and sparse matrices differently. Its cores are twice as fast as Turing's for dense matrix and four times as quick for sparse matrix that have all the needless weights removed. The upshot, per SM, is dense processing at the same speed - it has half the cores, remember - and twice the overall throughput for sparse processing.

https://hexus.net/tech/reviews/graphics/145342-nvidia-geforc...

Yeah, but ResNet does not have sparse matrices, so how could it use them? Post ReLU activations may be sparse, but I don't think that helps when used with a non-sparse Conv2d.
If anyone has access to a 3080, a full benchmark of hashcat would be fantastic. hashcat --benchmark-all
No full benchmark but this looks like it will be promising. https://twitter.com/hashcat/status/1306937641653465090
Thank you
Incomplete, but more details than the initial benchmarks https://hashcat.net/forum/thread-9511.html
Also jtr benchmark output.
It is quite interesting than on FP16 RTX Titan kicks RTX3080 ass and on FP32 it's the opposite.
The number on RTX Titan for fp16 is actually a bit foreign to me (3.5x faster than fp32). Not sure what's going on (maybe they tweaked batch size to fit more in GPU memory?). Need to wait more comprehensive ones.

(The older numbers looks more inline with what I remember and here is an alternative benchmark for RTX Titan showed similar: https://lambdalabs.com/blog/titan-rtx-tensorflow-benchmarks/)

You're using half as much memory for something that is vaguely O(n^2), so there's your roughly 4x factor speedup. But since things aren't perfect, it ends up being a bit lower.
They boosted FP32 performance but now FP16 is 1:1 to FP32 cuda cores
They look great, but I would be much more interested in time to a specific accuracy on the validation set. Images/second doesn't really matter when the the computation may not be exactly the same (FP16 vs FP32, sparsity, as it was mentioned in another comment)
Is there still some licences meddling with the use of gaming cards for deep learning?
From the GeForce software license:

"The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted."

https://www.nvidia.com/en-us/drivers/geforce-license/

This doesn't preclude use in a workstation, but they don't want you building a DGX A100 competitor using RTX 3080 or 3090 cards.

As far as I understand it, you cannot re-sell (rent) processing time on a gaming card from Nvidia for machine learning tasks. This is why you don't see Google and Amazon offering virtual machines with gaming cards. However if you're the end user, you can do whatever you want with the gaming card you bought (not that Nvidia can enforce this).
So is the single slot graphics card more or less dead?