There are many ways to go about this. Files like those keyfiles and encryption headers are extra sensitive because (a) they potentially provide access to everything and (b) losing them can block access to everything. Personally, I keep those types of files unencrypted in a directory that stays 100% offline (encrypted backups to external disks only). But there’s no reason not to back those files up to an encrypted online repository (where you trust the encryption). Just make sure that’s not your only backup of those files for obvious reasons.
A good practice to avoid painting yourself in a corner is to test your backups: Switch off your PC / server, put your mobile devices in a drawer (pretend they’re gone), borrow / wipe a cheap laptop. How do you access your backup files using just that laptop?
Good catch… and that’s why I keep up-to-date encrypted offline backups in two locations (home and office) always. That should be enough really, but I’ve been thinking about swapping one of those drives with a third backup at one of my relatives’ house from time to time, just to make irrecoverable failure even less likely.