How To Change Or Add New SSH Key for EC2

This is a documentation of how to change or add new ssh key for your EC2 instance if you lost, and maybe compromised your private key.

The gist of it is to add in a new key pair to the disk volume of the EC2 instance. Pretty straightforward! But how can you do it without  being able to ssh into the EC2 instance without the private key you just lost? You will need to attach the root volume of the EC2 instance to another temporary EC2 instance, which you can access with a new key pair, and add in the new key pair to the original volume from there.

Summon the NewKeyPair!

First, create a new key pair. You can either generate a private and public key pair on your own and import the public one into the AWS console, or create it from the AWS management console and download the private key that they generated for you thereafter. Should you go for the latter, make sure your browser is not blocking the download.

Blocked download | vic-l

For the rest of the article, the new key pair will be referred to as NewKeyPair, and the old key pair LostKeyPair.

Retire The Veteran

Stop your old instance. Do not terminate!

NOTE: Your instance root volume need to be EBS backed and not instance store as instance store volumes are ephemeral. They do not persist the data after power down.

Once it has successfully stopped, you will realise that its volume remains attached. That’s EBS for you!

We will come to detaching it in a while. For now, spin up a new server.

Katon: Summon-The-New-Server-Jutsu

Launch a new server with the NewKeyPair. This is a temporary server and can be any of the linux distribution.

Detach The Old Volume

In the volumes page, select the old instance volume and select Detach as shown. There should be no error unless your old instance is still in the process of shutting down.

Detach old EBS Volume | vic-l

Once it is detached, you will observe that its status has changed to available and its Attachment Information will become blank. Now it is freeee! Time to attach it to the new server and receive its new key pair.

Attach To New Instance

Attach the root volume to the new instance as shown.

Then select the device to mount on.

attach volume device | vic-l

I will set /dev/sdf as suggested. The other devices reserved for the root volume (/dev/sda) and instance store volumes (/dev/sd[b-e]). More information on the device naming in AWS EC2 can be found here.

Run the command lsblk to see the new volume mounted. Note that the linux kernel has change my mount point from sdf to xvdf as noted in the warning callout in the image above.

lsblk | vic-l

Mounting The Volume

You would not be able to use the volume right away after attaching without mounting the volume in the system. Mounting will tell the EC2 instance how to access this new device via its list of directory. This will require setting up a mount point. Run the commands below.

sudo mkdir /mnt/tempvol
sudo mount /dev/xvdf1 /mnt/tempvol

These commands will mount the root of the device to the directory named /mnt/tempvol. You can change directory into the volume and see that it contains content from your old server.

From the image above, you can see that the authorized_keys file containing the old public key is placed in /home/ubuntu/.ssh directory relative to the mount point. The new public key pair exist in the /home/ubuntu/.ssh directory in the absolute path, which exist in the root volume of the new instance.

Adding The NewKeyPair To The Old Volume

Eventually, we want to use the new key pair to access the old server, with the content of the old volume, just like the good old times. To do get, add the NewKeyPair to the ssh folder of the old volume.

Realise that this is now possible because of the attaching and mounting of the old volume to a new instance which can be accessed due to a fresh setup and key pair creation.

I have used the append operation, >> instead of the overwrite operation, which is a single >. This is not necessary. It is up to you to decide if you want to get rid of the old key pair or not, depending on your situation.

If you lost your old key pair, feel free to overwrite it. There is no point hoarding it, and Marie Kondo can’t help you declutter software.

Attaching The Volume Back

Next, you can shut down your new server and attach your old volume back to the old instance. Remember, the EC2 instance will not be deleted and is still available if you chose stop instead of terminate when shutting it down initially.

attach volume back | vic-l
Mount the volume, this time, to /dev/sda1.
attach to sda1 | vic-l

The reason to mount it at /dev/sda1 is because we need to give the instance back its root volume for its boot operations. If we were to mount it to another device, you will see this error when starting the server because no root volume is detected.

error starting old instance | vic-l

Back To The Past

Now you can try to connect to your old instance after it has started up.

NOTE: you will see that the connection instruction is still mentioning the LostKeyPair. Even if you had overwritten your ssh key pair to the new one, this wrong instructions will still persist. Of course, you should connect with your NewKeyPair.

Connect to old instance again | vic-l

To ascertain that you have the old public keys, head dow to the <code>~/.ssh</code> directory and see that the changes you made on the new instance via attaching and mounting of the old instance’s volume has persisted.

Now, we have successfully added a new key pair to the old instance, and we can use it to ssh into the old instance form now on, even though we had lost our original key pair that was used to create it.

Leave a Reply

Your email address will not be published. Required fields are marked *