If your software is CRUDe or has many alternatives/examples on the web, altering it with an LLM is easy, since the model already knows what you're working on.

When the math you're working on bespoke, and the optimizations you do are intricate, these models and agents fall flat on their face, because there's no "a set of widely used optimizations" (i.e. an open repository of very nicely optimized code for cases X, Y and Z) to apply on top of the codebase.

By bespoke, I mean, there's only one academic paper written about it, and it's written by me and my supervisor.