As a developer I am keen to see some compilation benchmarks. Unfortunately those kinds of benchmarks are almost never included in such reviews. Instead there are many gaming benchmarks which purpose is not exactly clear for me, after obviously gaming is not the primary target market for R9 3950X.
back
7 comments
I believe that some of the YouTube channels have started including compilation benchmarks. GamersNexus [1] and Linus Tech Tips [2] both do!
[1] https://www.gamersnexus.net/guides/3460-new-cpu-bench-method...
In addition, Level1Techs is more developer/sysadmin oriented than consumer oriented : https://www.youtube.com/channel/UC4w1YQAJMWOz4qtxinq55LQ
And in addition to that, we have Open benchmarking and Phoronix for that. :)
Likewise (I 2nd L1Techs btw). Having 16 cores/ 32 threads will greatly aid testing server applications. Making multi-threading & locking issues much more apparent (Amdahl's law) than they would on a slightly older CPU with only 4-6 cores (a i7 6700k in my case).
Normally we at AnandTech run a Chrome compile benchmark, but for whatever reason it wasn't running properly on Win 10 1909. When I get a chance to debug (55k miles of travel over the next four weeks), I'm going to see if I can fix it and expand that bit of our testing.
"Failed to compile Chrome" is a pretty huge caveat in which, I guess, your readers might take an interest.
Chrome has its own Windows builders as well on each platform.
Note that the GamerNexus review of the 3950x didn't include their compile benchmark test, at least on Youtube.
As a developer, other benchmarks I would like to see:
* compile benchmarks for different mainstream languages (Java, C#, etc.)
* IDE related benchmarks (i.e. how long does it take to index a large solution/workspace)
* source control conversion tests (svn to git). Obviously not something that happens everyday, but I've done it at two different jobs, and generally when it happens you are often converting many repos as an organization shifts its policy.
* maybe some VM/docker related tests
I'm probably asking for too much, but working on a micro-service system even with enough RAM my system becomes less responsive when I'm testing the interactions between services on my local machine. I'm also limited on the number of Java projects I can add into a single workspace in Intellij. I've been forced to open each project as a separate workspace and in a separate window.
Granted, I'm working off of a dual-core i7 laptop (with 32GB of RAM) but I want to know what kind of upgrade it would take for those problems to go away.
> Instead there are many gaming benchmarks which purpose is not exactly clear for me, after obviously gaming is not the primary target market for R9 3950X.
Well, it must be said that the 3950X has the fastest single-core performance of AMD's lineup, so while it is overkill from a core-count perspective, it's still technically their best gaming CPU.
That makes two of us! Always wanted to see reviews/benchmarks from the angle of a software developer job.
I don't care how about video transcoding or most of the metrics such article show. But how long does my IDE take to index a big source tree? You bet!
Got any standardized methods that could be automated/scripted under a clean Windows environment and doesn't require internet access / licensing? Give me a shout - ian@anandtech.com
.NET Core is open-source and free, with an offline SDK installer. [1] There are plenty of large open C# code bases, like the ASP.NET web framework [2], that you can use to compile and get a good performance score.
And like the other comment said, you can probably get complimentary copies of the IDEs from Jetbrains to run benchmarks with. They're also scriptable which helps.
1 .https://dotnet.microsoft.com/download/visual-studio-sdks
> Got any standardized methods that could be automated/scripted
I'm sorry, I wish I did! Interest in hardware performance is only a relatively recent interest of mine. I especially think the IDE tests would be a bit challenging as I'm not even sure how to instrument or measure those. I might reach out to the Intellij folks to see if they have any ideas..
> under a clean Windows environment
Ah, shucks. I haven't touched Windows in years :). That's the tricky thing: according to Stack Overflow's surveys, only about half of software developers use Windows. The other half are split evenly between Mac and Linux. Covering all three platforms would likely be a challenge.
> Give me a shout
Thanks! I'll do some research, and if I have something more concrete I'll be sure to pass it along.
Honestly something like a Linux Kernel compilation would be the most straightforward, static and consistent method to test it out. Just have a basic environment without overhead (like Alpine Linux) with GCC or LLVM/Clang sitting on a USB key, use GNU time to measure the execution of your command, redirect to a log file and you're done.
user@host$ (time make -j[insert number of threads]) 2>logfile.txt
Linux Kernel would probably be good. WSL could probably do it without much trouble though I don't know how WSL affects performance compared to native Linux.
Compiling Visual Studio Code Typescript could be an interesting target too.
There should be a large open-source C# and/or Java project out there that can be compiled as well.
Is Typescript compilation multithreaded?
The source code is in .ts, and appears to use Node child processes [0], so probably yes. I'd be curious to see the kind of resource usage is involved building TSC.
My limited experience using Node worker threads (to compile small React apps with WebPack in parallel) showed lots of overhead and memory usage for just starting the build (Node and library instances, can't say in what proportion), although it is still worth the expense.
[0] https://github.com/microsoft/TypeScript/blob/master/scripts/...
A good question, though I will point out that either way benchmarking would still be useful.
Maybe anandtech.com could create / pay for such tests?
As already said by others, Phoronix has some compilation benchmarks.
I have recently bought a 3900X. Some things about compilation I would note is that it heavily depends on your programming language and tooling. And that you would note is that you get most benefits on clean builds of big projects, which scale very well per core. On incremental builds or smaller projects it is however not uncommon to see < 25% of the CPU being used. That is especially with Rust, where a single compilation unit (crate) is compiled in a single-threaded fashion. Might be better with C or C++. But then again linking might also be blockers.
It's all nice and fast, but on day to day use you likely won't see a 100% speed increase compared to a 3600.
Beside clean builds is Git branch checkouts. I hit this at least twice a week.
Finish a feature branch and push it. Check out master. Pull master. Make new feature branch. Get an urgent bug fix request, switch to release branch, make a bugfix branch. Now go back to the new feature.
During this I may end up doing large rebuilds because Git changes modification times even if you end up back with the same header file you started with.
On a Thinkpad T540 the rebuild can take 30 minutes. It's less than 5 on a 3900X.
Use a build system that works with input-hashes.
djb's redo is an example.
Do you know about `git worktree`? You may not want a separate one for each branch you work on, but at least one for master and one for release would help you I think.
There are problems with worktree. But yes, I often do use multiple checkout directories. When you do the git clone using the --reference option it gets most of the objects from the referenced directories.
But this still results in slow compile times because the less used release directories get stale, have to be pulled up to date and often mostly rebuilt anyway. If I remember to actually use the second checkout directory, it does save on needing to rebuild the master/feature branches afterward.
ccache solves this fairly well if you're on *nix.
Another potential area might be unit testing. As an example, I've been running a test suite on a C# github project (xUnit and mocking libraries) with ~1.5K tests. The CPU is at a solid 100% cpu usage all the way through.
In higher-level languages, running tests often takes considerably more time than building. A prosumer CPU like this would bring nice day to day improvements.
GCC at least goes very fast on the new Ryzens, due to the huge l3 cache. That may translate to many compilers. Rust (which uses llvm on the backend) also compiles very quick.
For the 4000 series, they're planning on sharing the L3 caches between CCD's/ chiplets rather than per CCX. It will be interesting to see if the more cache availible per core, and more cores being able to access the same cache will improve the performance.
The speed AMD has raised caches & core-count lately is insane. When the max for mainstream CPUs was 8MB (and 4 cores) until only a few years ago. (Things seem to have stopped when quad-core launched, the Intel Q6600 was an amazing CPU at the time.)
I've been a computer enthusiast for over 20 years and didn't expect to see advancements to return like the good ol' days (Moore's law) to ever return. (I don't think Intel did either, halving their prices pre-launch is unprecidented!)
Yes I look forward to the better cache system in the next generation!
Do remember that part of how AMD can get so much L3 on there, is due to the chiplet design, which also adds a LOT of latency to ram accesses. Almost all of the time, it seems to be a net win though, and with the next generation it will only get better.
It would be interesting if we moved back to the final layer of cache being off-core, likely within the IO die or attached as additional chips within the package. That should decrease the core size, improve yields, and allow for differentiation in L3 size as well.
but it would add latency :(
but maybe an L4 cache, off core, as additional chips!
At some point both higher latencies and deeper cache hierarchies seem inevitable: http://www.ilikebigbits.com/2014_04_21_myth_of_ram_1.html
Though, perhaps 3D integration can stave that off for many more generations.
Interestingly, Broadwell-S and Broadwell-R did this using the "Crystal Well" L4 cache.
However, DDR4 memory now provides more bandwidth than the L4 could (although probably at a higher latency).
Normally I run a Chrome compile benchmark, but for whatever reason it wasn't running properly on Win 10 1909. When I get a chance to debug (55k miles of travel over the next four weeks), I'm going to see if I can fix it and expand that bit of our testing.
Agreed. At least it's part of the phoronix benchmark, and openbench has some results:
https://openbenchmarking.org/showdown/pts/build-linux-kernel
I'd love to hear of a more richly supplied crowd sourced database.
I'd love to see more container-oriented benchmarks, particularly with docker-compose and/or minikube.