My guess: As you go to faster circuits, parasitic capacitances and a lot of high frequency nuances become even more critical. These effects are relatively more difficult to design for, and not a matter of just recompiling, unless you are making a slower chip (but why would you do that using an advanced node?). So everything is a bit more difficult: metallization, distances between different circuits on chip, etc.

In addition to this, manufacturing design rules get more stringent and complicated, which result in more headaches for the designers. This is not only from the HF standpoint but things like narrower voltage windows, high power dissipation from all those extra transistors, etc.