I think it's debatable whether the module system really is complex or just different from what newcomers are used to (and by now I've grown pretty accustomed to it). But in contrast to most other language features, where it was clear what I was getting in return for the steep learning curve, the module system seemed overly complicated at the time for no real benefit. Not a big issue by any means and I would choose Rust with its module system over the alternatives most days of the week, but it is one tradeoff that to me at least seemed orthogonal to the other borrowing-related complexities.
(What is more worrying to me nowadays is the whole async story. I do hope that some of it will get better once certain features land and it is certainly an area where some additional complexity is unavoidable, but it is the only part of Rust that I dread touching despite heavily using async in a moderately sized personal project due to the need for WASM + IndexedDB, simply because lifetime issues become much more tricky once async and either traits, recursion or closures are involved. By now I am consciously trying to limit any async parts of the program to a simple and stupid "Rust-light" style without any "fancy" features such as traits or closures, which does not feel like a proper solution. So yeah, in general I agree with you: Rust is certainly complex, but for the most part not unnecessarily so.)