Plain C++ can easily lose to JITted scripting languages. Nothing new or weird in that. Remember, we're comparing native code to native code.
back
3 comments
Yes! But it's important to keep in mind the sort of optimizations that JIT enables: monomorphic dispatch, type specialization, etc. And also what it's bad at: computationally expensive or memory-intensive operations, like graph-coloring register allocation.
So generally speaking, we'd expect JIT to shine with certain branchy or unpredictable code, but static compilers to shine for basic blocks. Since the hot spot of the code in question is one big basic block (no if statements, etc.), where even the loop trip count is known statically, it would be surprising in this case if a JIT outperforms a static compiler.
Yeah, it might be worse at allocating registers, but on the other hand it can often free registers as well.
Static compilers just don't have the runtime information to take advantage of. Some maybe, by using profile guided optimization. JIT can always have profile guided, and adapt just for that particular case.
Right, but JITs only have the runtime information after a slower warmup period, and even then recording and exploiting this information has a cost too. The real divide here is between client/server: fancy JIT techniques can benefit long-running apps on beefy servers, but won't help a Java app launch faster on a smartphone.
There's absolutely no reason why there needs to be a warm-up period. You can cache the JITted binary code to disk. CPUs really don't care how the code was produced.
A sampling profiler can be pretty cheap. Just record stack state every n milliseconds, no need to instrument everything.
Actual JITting can be done on those other cores that would otherwise be idle anyways.
Maybe - it sounds tricky though, since optimizations can be rendered invalid across runs (e.g. a plugin may be loaded that invalidates a monomorphic dispatch optimization). So you must track and re-validate the assumptions underlying the optimized code before it can be used. Also startup code is often executed once, and so is a poor candidate for expensive JIT optimizations. That said, in principle it seems like one could do as you say - it's just hard.
Empirically, the industry seems to be moving from JIT to AOT, at least for clients. .NET -> Ngen, Dalvik -> ART, JavaScript -> asm.js, etc.
(Oh, and it's a suspect assumption that there exists other cores that "would be idle anyways!")
AOT doesn't exclude JIT! Doing both is the winning combination. AOT will ensure fast start-up time and JIT will ensure optimization to runtime conditions.
It's tricky I'm sure, but it'll happen.
Not true actually theres always a JIT overhead for the compiler.
There's also a significant overhead for not being able to target exactly the CPU model and parameters for the call at compile time. Unless of course doing profile guided optimization just for that CPU model and use case.
I've been playing around with an idea for a long time to write a JIT for native code. I think it's possible to speed up most native code by JITting it. Sadly it would take much more time I can afford to spend. The principle is sound, though.
It'd be possible to eliminate a lot of computation at runtime. Remove a lot of branches, use CPU model specific instructions when a suitable pattern is detected, etc. Functions could be simplified to constraints, function calls could be dynamically inlined. Spilled stack variables could be allocated in registers. Calling conventions optimized into passing more parameters in the registers. On register starved 32-bit x86, registers for parameters with effectively constant values could be converted into instructions with immediate values, saving registers for actually changing data.
There's also no reason why you can't cache previously JITted code into a file.
Not only does JITting normally only occur once (something that microbenchmarks take into consideration by "warming up" the method), but if it does occur an Nth time it's because the runtime has acquired profiling information to do a more intelligent optimization pass on the method: far exceeding the quality of assumptions/heuristics made during the first JIT pass. As far as I know Hotspot implements PGO JITting.
Static compilers can do profile guided optimization, but this only takes into account the static scenario that you profiled - not the real world scenario that the program is encountering. Your PGO is only as good as your profile.
In theory, not only is your statement untrue - but reality can turn out to be the exact opposite.
Easily?
Do you have any benchmarks to back that up?
My sense is that JITet platforms lose most of the time still, but that the JITted platforms are getting closer, and occasionally win.
Yeah, I do. Kind of.
Static code carries a lot of overhead in some cases.
SwiftShader is a specialized JIT. It generates the code at runtime. It beats any C-code by about an order of magnitude.
https://www.transgaming.com/swiftshader/faq
All fast RegExp matchers do JIT compilation. Like PCRE. Native code just can't compete. IIRC, about an order of magnitude advantage.
Firewall rules can benefit from JIT by an order of magnitude. For example: https://wiki.freebsd.org/SummerOfCode2014/ConvertingIPFWRule...
> All fast RegExp matchers do JIT compilation. Like PCRE. Native code just can't compete.
I thought that most "native" regex engines basically amounted to small interpreters written in a compiled language. I'd be surprised if the languages that actually compile their regexes to native code are not competitive.
Yes, that's exactly what I meant. Those "small interpreters written in a compiled language" have no chance against Just-in-Time compiled code.