Why Ice Lake is Important – a bit-basher’s perspective

I hope you have a comfortable bed, then. It will be a while before there are machines deployed across the whole range (client laptop, client desktop, server, network edge) that are SNC. Atom isn't even supporting AVX2 in the short term either.

Daniel Lemire, who I work with on simdjson, wrote this on throttling: https://lemire.me/blog/2018/08/13/the-dangers-of-avx-512-thr...

I don't know specifics (I'm ex-Intel a good bit before SNC, and wasn't a hardware guy) but my impression is the clock throttling impact gen-on-gen goes down. If you're actually getting good value from AVX-512 instructions rather than chucking in 1 in 1000 instructions for the cool factor, you're probably fine.

"Execution port Twitter" seems divided on what the extra shuffle port will do. I'm cautiously optimistic.

Integer division is never really on my radar, but yes, that's exciting.

LEA is very useful, but not something I spend a huge amount of time thinking about. It would be interesting to see whether the penalty for a "slow LEA" is relaxed.

You're right that the Skylake-X processors have AVX-512, but they kinda push the 'consumer' envelope. There's also no consistent branding for 'having AVX-512 or not' (they have happily adopted i7/i9 for products that don't have it). These are really obviously rebadged Xeons. Nothing against them, but it's a tiny niche.

Using these instruction sets usually requires explicitly writing code using C language intrinsic functions, which is a reasonable compromise between transparent automatic usage in C++ and needing to write assembly. On the positive side, most compilers have full support for these intrinsics and the compiler can even do compile-time evaluation for a few of them e.g. some intrinsics are treated as 'constexpr' in C++ by the compiler, but the compiler won't automagically use them for much and is generally poor at identifying ordinary code that can be reduced to these intrinsics. So for designing algorithms that take advantage of the advanced instruction sets, you are mostly on your own. Writing C/C++ code with heavy use of intrinsics isn't hard to learn or particularly opaque, just arcane.

These instruction sets can offer very substantial speedups for code paths when applied judiciously. You do need to check the CPU capabilities at start up time for practical software, you can't assume the environment supports it even when these instructions have been around a long time. Some had very uneven implementation timelines across CPU product lines. Even for explicitly high-performance software, I invariably would get bug reports because someone tried to run it on a 10+ year old machine that did not support the instructions I was using. There are standard open source libraries for doing this feature detection using CPUID.

Toolchain support, at the minimum, often consists of having a compiler flag to allow the use of intrinsics for the new architecture - pretty much the opposite of what you're hoping (automagical support).

There's a fair bit of work where people try to autovectorize code. I'm happy that folks are pushing this work, but, well, I drive a manual (in this regard).

It does happen that code does auto-detection and skips to a particularly compelling platform-specific optimization path, but it's a huge pain. It's unlikely that someone will optimize for each and every variant, but they might draw a line here and there through the myriad of architecture choices and offer something.

Sorry that the story isn't better. I generally hand-code stuff - not in asm but with intrinsics (which is getting a lot of the benefit of asm, but without doing all your own codegen for the boring bits, scheduling and register allocation).

GPU HW dude, here: I’m not sure of AVX-512’s current fma16 story, but it’s predecessor (LRBni) was terrifyingly good at fma32–much better than contemporary GPUs. The problem with those parts is total power displacement (TDP). The market is really geared to mobile; so, so that became less of an issue. INTC-for-Graphics is hampered by the lack of deep, dedicated latency hiding fixed-function HW. Otherwise, INTC is probably a strict win over any discrete GPU.

(Disclaimer: I wrote software rasterizers, drivers, and compilers for Larrabee.)

That's exactly the kind of thing I don't really know about. I'm not hostile to that sort of work, I just can't really add any information to it. There's clearly a huge market to building the Biggest Baddest Linear Algebra Engine out there, but I hope that other uses of SIMD don't get neglected.

Ice Lake's Wikipedia page mentions that Intel is adding hardware mitigations to Spectre and Meltdown.

Ach, you're right. Just a mistake. I tend to forget the existence of any operations that are (a) in MIC and (b) not available for bytes, so this was a double-whammy and even after writing that sentence I forgot that I was contradicting myself.

It partially depends on the design of the database engine. Some classic designs are not that amenable to it but newer designs do tend to be. Many database operations (like operations on large bitmaps) are effectively memory-bandwidth bound, unless they are running completely out of close CPU cache. Some scenarios where vector intrinsics are commonly used:

- Using these instructions to encode/decode compressed data structures where evaluation would otherwise be memory-bound, trading the "free" CPU time to reduce memory bandwidth consumption. Large bitmaps are amenable to this.

- Increasing throughput of operations that are CPU bound so that they become (closer to) memory bound, recognizing that the memory bandwidth available to CPU cache-friendly structures may effectively be much higher than RAM bandwidth. Format parsing and cache replacement is a common use case for vector intrinsics in databases for this purpose.

- Reducing branch prediction penalties for data structures that don't play well with branch predictors. Commonly used hot path radix-trie structures often have this issue. Some of the intrinsics don't have a direct scalar equivalent and redesigning around them can significantly reduce branching pressure (both number and predictability).

- Scan operators for vectors of high-level data types, ironically because they allow you to efficiently use a more complex representation than a simple vector. Pages in databases are large enough these days that this is worth doing and you'll spend much of your CPU time here anyway.

I have not (yet) built an operational database kernel that uses AVX-512 intrinsics, though the designs are setup for it, mostly because support is still insufficiently common at the moment. I think the CPU throttling aspects of vector instructions are less of a concern for database engines because good designs tend not to have clock rate as a major throughput limitation in practice, though I've never actually measured it.

I think it can help. Traditional DBs generally worked on the assumption that if you were scanning through records one-by-one you'd already lost, so naive ideas that you can just write Really Good SIMD won't help. But there are plenty of smart people who understand this well and can write SIMD code for what remains.

Scanning non-DBMS stuff can benefit a lot from good SIMD (although writing code that's tremendously clever that allows you to run faster than memory bandwidth permits is also fairly self-limiting; the faster you go, the less likely anything can keep you fed).