Yup. I've seen 40x speedup over C/C++ code just by using SIMD (SSE/AVX) and eliminating the branches other than loop condition. It was possible to process up to 16 results in parallel (using just one CPU core), with about same number of cycles as the C version took to process one, while getting rid of the penalties from mispredicted branches. Data dependent branches are very expensive. They often have nearly 50% branch mispredict rate. If the branch is data dependent and thus unpredictable, it's usually much faster to compute it every time regardless and simply mask to ignore unwanted results than to eliminate computation by branching over it. Depending on the CPU model, with SIMD you can compute roughly up to 100-400 floating point ops (in SIMD vectors) and 200-800 integer ops in same time as one mispredicted branch takes. In those 14-20 clock cycles! Haswell executes up to 16 floating point ops per core in one clock cycle using SIMD. This is, assuming you don't get data starved, of course. Or have pipeline stalls due to unbreakable too long dependency chains, etc.
One thing compilers just can't get right is when there's a large switch-statement in performance critical code. Compilers can't seem to be able to make intelligent decisions about register allocation in that case, and behave as if you had true function pointers (=disable optimizations). In that particular situation you can gain 2-3x performance over a compiler just by dropping to assembler.
Compilers are pretty good in most cases, though. I wish they were smarter at eliminating branches.