In any case, the issue here with debug builds does not seem to be due to memory access nor copy-on-write, but simply because of ARC sprinkling the accesses to the array with release/retain pairs.
The reason it works ok with optimizations turned on, is because Swift will not remove unnecessary release-retain in -Onone.
This was not a great problem in ObjC, since it only affected ObjC method dispatch - which you were unlikely to do in a tight loop. Plus, ObjC ARC actually emits retain/release pairs differently.
The issue with ARC emits was discovered with the very first beta, but as of yet there has been no fix.
The problem here is probably this:
1. In order to perform a local optimization on ARC retain/release, even at -Onone, you need to change the retain/release emits to the style used by ObjC.
2. But that retain/release often incurs an unnecessary autorelease, which would prevent even an optimized Swift build from eliminating certain retain/releases.
In other words, they either get significantly better -Onone performance, or slightly worse -O/-Ounchecked performance.