For the VM, follow the same OS hardening standards you would apply in a non VPS environment. Each OS distribution has its own NIST, CIS hardening guides. Do not use pre-built images created by someone else, as you have no idea if they are tainted. Build, patch and harden the image in your own continuous build environment.

Each VPS provider have their own site specific means of locking down their API's and web consoles. Speaking in generic terms, limit API access to known trusted networks when you can. Limit the lifespan and capabilities of API keys. Rotate API keys, just as you would passwords or ssh keys. Delegate the minimum permissions required for people to do their job. Audit accounts and delegated sub-accounts. Set up multi-factor authentication for non API key logins. Set up email alerts for account changes.

In high level terms of networking, do not allow unfettered outbound access from your VM's unless there is a specific need to do so. Either set up your own traffic logging at the host level, or determine if your VPS provider have the ability to log traffic.

In terms of payment methods, use a virtual account number (virtual credit card) that is tied to that vendor and has a currency spending limit. Not all banks have this capability. Ask yours to add it. When possible, set upper limits on the number and sizes of VM's your account is authorized to create. Ask your VPS provider to add this capability if it does not exist.