Continuously getting locked out from EC2 instance. No supported authentication methods available

Question

I am getting continously locked out from my ec2 instance time and again. I have to take its root volume and attach it to some other instance (mostly new instance) to prevent data loss. I am not able to figure out what is causing my ssh being denied. It works for sometime and all of a sudden ssh is denied. I have tried

1. Comparing the authkeys with my login key. It is same.
2. Copying the key from a ssh working instance to this volume and re-launching. Same error.
3. Did some tweaks in sshd_conf and ssh_conf but to no success.

I am completely stumped here, any viewpoints on this. I have been working on AWS for sometime now and never encountered this problem, the only change being the organsation (and the network) has changed from where i am working from? Can their network be generated some kind of DOS attack which is just shutting down my ssh access? My team does not have an official AWS tech support plan, hence here.

Answer 1

Although i have not found out what leads to this. But i am finally able to login into my instance by following these steps: 1. As you are logged in create a user in the instance and give it a password.

$ adduser user1
$ passwd user1

2. Enable password authentication in /etc/sshd_config file. 3. Login with the new user.

Interestingly, this user does not gets locked out as the centos user.

Answer 2

One way I get locked out while developing is by changing the permissions on any of the directories or files associated with the keys used by ssh. This happens to me when I untar files or run some installation script while initializing the machine that touches the ec2-user account.

Make sure the permissions are like:

1> ls -ld $HOME $HOME/.ssh
drwx------ 15 ec2-user ec2-user 4096 Aug  1 13:15 /home/ec2-user/
drwx------  2 ec2-user ec2-user 4096 Jun 10 11:18 /home/ec2-user/.ssh/

2> ls -l $HOME/.ssh/authorized_keys
-rw------- 1 ec2-user ec2-user 385 Jun 10 11:18 /home/ec2-user/.ssh/authorized_keys

If not, and you are logged in as ec2-user, you can change them with

sudo chown ec2-user:ec2-user ~ec2-user ~ec2-user/.ssh ~ec2-user/.ssh/authorized_keys
chmod 700 ~ec2-user ~ec2-user/.ssh
chmod 600 ~ec2-user/.ssh/authorized_keys

To protect myself against my own mistakes, I create a new user and copy the ec2-user key over. That account is used as a backup and not part of any script. Once I tested everything, I remove the account.

Answer 3

Thumb Rule: Not finding a root cause is Not a solution

Now from your inputs What we get is

1. You get locked out while using SSH login (passwordless)
2. You are able to login if its password login

what I think might have happened is that your private key file format might have been changing (one consideration is EOL) if you are using a windows / mac or linux editors or switching between them for SSHing into the remote.

So do a "cat -e ~/.ssh/id_rsa" and check the contents if you see a ^M at the End of the line , then most probably you have edited the private file with a CRLF editor. Do a "dos2unix ~/.ssh/id_rsa" to convert the file to LF format

Check this https://unix.stackexchange.com/questions/32001/what-is-m-and-how-do-i-get-rid-of-it