I haven't run into this yet. My test failures have either been real or have been triggered by an (intentional) breaking change. The later does require updating the tests, but I think catching the real bugs is worth that tradeoff. I haven't found spurious failures to be a big problem (I've had a few but rewriting the tests that have this problem has eliminated it for me). My codebase is a pretty modular rails app, which I think helps with this.

I have worked on other projects where flaky tests are a problem (and generally cause developers to ignore and submit anyways), but so far I've avoided that. My experience with flaky tests is that they're typically due to poor modularity or to subsystems that other teams can modify. This project exists in a monorepo and I'm the sole developer so that's not a problem here.