You are making multiple assumptions about "legitimate users", which biases against various groups:

1. They have (or are willing to use) an online payment source (credit card, paypal, etc.).

2. They have (or are willing to spend) a disposable income.

3. They trust you to get their money back.

Also, you're going to deal with a huge amount of customer service as every single person argues with you that they didn't violate the terms and deserve their money back - that's going to cost way more than any fees/interest you'd earning from the escrow.