I've been using a similar setup for probably 8+ years now. It started out with an ownCloud instance on a Raspberry Pi B+ (but has since evolved to nextCloud and a VPS, which I also use to run other things). I've been incredibly happy with it. I didn't have to worry about moving my contacts over when I switch to a new device (whether that be laptop or phone) – I just have to add the synchronization to the new device. This, plus calDav (I just `khal` and `todoman` for CLI calendar and to-do lists) have been excellent.