That sounds like it'll lead to human-driven reward hacking [0]?

[0]: https://en.wikipedia.org/wiki/Reward_hacking