GPG
To export your private key, do:
gpg --output private-key.asc --armor --export-secret-key username@domain.com
Then you may encrypt it with a symmetric key (but if the private key is password protected, it will already be exported in password protected format)
gpg --symmetric -o private-key.asc.gpg private-key.asc
When restoring the backup, first decrypt the symmetric encryption:
gpg --decrypt private-key.asc.gpg > private-key.asc
Then import the private key to your keyring
gpg --import private-key.asc