But overall -- I work for an insurer. Having to document how something was done a year+ after it was done to convince auditors/insurers/large clients is a LOT easier if you can point to a specific piece of hardware vs. a bespoke process. The downside of not having tamper-evidence or tamper-response (ideally) at the hw level is you need to build a lot of other controls around it, and that is expensive, impractical, and difficult to document in a lot of environments. e.g. in a datacenter it's unlikely you can enforce "two man from security/senior executive team in the cage"; it's probably going to be (at best) two ops staff, often contractors, and two-man rule isn't actually that widely used outside CAs themselves. The use cases I'd like for this are application logic, not traditional CAs (who do need FIPS). Add to this lifecycle management of the hardware from production to pickup/delivery to provisioning, pre-deployment storage, deployment, operation, routine audit, replacement, audit, and decommissioning, and it's way easier to put protection into the module. Most of the time I've just seen hardware protection inherited from single-chip vs. module-level protection created, but if there's no certified IC available, module protection it is.
Actual FIPS certification would be nice (or a better standard), but challenging in deployment because almost every vendor FIPS certifies with a specific application load and as soon as you run custom applications (which is badly supported in the HSM world), you lose certification anyway. "FIPS rated hardware running a non-FIPS load, audited in appendix A" is insurable, but ideally there would be something better. 99+% of HSM deployments are just used as dumb key signers which are trivially exploitable when you pop the host, though, so I'd certainly take the non-FIPS option.
(Today, the state of the art is pretty much SGX or another TEE on the host running custom application logic to evaluate and instruct signing which talks to the FIPS HSM to do key control and actual signing, e.g. what Anchorage does.)