You can create an SSH key pair with SSH keys from the Azure portal. However, it will save only the public key, not the private key. If you are the only administrator, you can save the private key on your machine or any physical drive. However, if you aren’t the only admin, you need to store the SSH private key.

The best way I found is to store it in an Azure KeyVault as a Secret, NOT as a Key. If you store it as a Key, you can only download the public key, not the private key. You need to get the private key when you want to SSH your Virtual Machine.

I assume you already have the SSH key pair. If not, you can follow another post in our knowledgebase.

Now create a Key Vault from the Azure portal, it’s a straightforward process.

You need to store your private key now. Use the following command to store in the Key Vault.

az keyvault secret set --vault-name <Name of your Key Vault> --name <Name of your Key> --encoding ascii --file <Name of your Private Key file>

When you need the key again, you can download it with this command:

az keyvault secret download --vault-name <Name of your Key Vault> --name <Name of your Key> --file <Filename to Save>

Also, you need to change the file permission of the downloaded file before using it in SSH

chmod 600 <Filename to Save>