Most devices offer a backup mechanism where the key material is encrypted and can only be imported in a same type device. For HSM'S this is called wrapping a slots key material with a wrap key. With gnupg you are allowed to export the key, or generate is on your computer and transfer it to the device.

Some services allow you to configure multiple separate keys, so different private keys. Lastpass for example.