Re-reading my previous post, it was perhaps a bit too negative. In practice you can get pretty good security fairly easily. Here are some steps I would take.

Most VPS hosting providers already have pretty good physical security - secure buildings, redundant power and network connections, lots of monitoring. The main risk by far comes from the humans using the installed software - accidental data deletion, clicking links in emails, responding to phishing phone calls, using weak passwords and so on. So my first tip is to make sure everyone is trained, and are using things like 2FA and password managers (not memorable or shared passwords).

Check that data is encrypted "at rest". Surprisingly few VPS providers use (or even allow) full disk encryption. That's because web servers are rarely "at rest", but they forget about backups. Backups really should ALWAYS be encrypted if you're serious about protecting user data and privacy.

Check that VPS setups are up to date - not just patched but following best practice. I very commonly see out-of-date security setups, by which I mean setups that were state of the art 10 years ago but are now totally irrelevant because the hackers have moved on. In particular, default setups often use obsolete ciphers and allow unencrypted connections (http instead of https), and omit http headers that deflect "XSS" and "SQL injection" attacks (which are still very common).

Many providers fail basic compliance checks like NIST, HIPAA and GDPR guidance. That's usually because they're not in the EU or the US or wherever, but compliance doesn't have to be compulsory to be useful.