Passwordless ssh with paramiko fails to authorize

Question

I am having trouble getting authentication working with paramiko SSHClient. Trying to go from one virtual machine out to another box on the network. The general idea is that I create a public/private key pair, ssh into the client using a password given, take the clients public key and add it to my known_hosts. Place my public key in the clients authorized_keys. Close that connection, and then try reconnecting without the password. It fails in the reconnection. I am using paramiko 1.15.2 and python 2.7.10.

The code goes as follows from this tutorial: http://www.minvolai.com/blog/2009/09/How-to-ssh-in-python-using-Paramiko/how-to-ssh-in-python-using-paramiko/.

import paramiko, StringIO, os    
pkey = paramiko.rsakey.RSAKey.generate(1024)    
pub_key = "ssh-rsa %s" % (pkey.get_base64())    
file_obj = StringIO.StringIO()    
pkey.write_private_key(file_obj)    
priv_key = file_obj.getvalue()    
server, username, password = ('host', 'username', 'password')   

ssh = paramiko.SSHClient()    
parmiko.util.log_to_file(log_filename)    
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())    
ssh.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))    
ssh.connect(server, username=username, password=password)   

sftp = ssh.open_sftp()    
sftp.get(remote_path, local_path)    
sftp.put(local_path, remote_path)    
sftp.close()    
ssh.close()

key = StringIO.StringIO(priv_key)    
privkey = paramiko.rsakey.RSAKey(key)    
ssh.connect(server, username=username,pkey=privkey )

This is the debug log that I get:

DEBUG:paramiko.transport:starting thread (client mode): 0x728ac950L    
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_5.3)    
DEBUG:paramiko.transport:kex algos:[u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1'] server key:[u'ssh-rsa', u'ssh-dss'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'[email protected]'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'[email protected]'] client mac:[u'hmac-md5', u'hmac-sha1', u'[email protected]', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'[email protected]', u'hmac-sha1-96', u'hmac-md5-96'] server mac:[u'hmac-md5', u'hmac-sha1', u'[email protected]', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'[email protected]', u'hmac-sha1-96', u'hmac-md5-96'] client compress:[u'none', u'[email protected]'] server compress:[u'none', u'[email protected]'] client lang:[u''] server lang:[u''] kex follows?False    
DEBUG:paramiko.transport:Ciphers agreed: local=aes128-ctr, remote=aes128-ctr    
DEBUG:paramiko.transport:using kex diffie-hellman-group14-sha1; server key type ssh-rsa; cipher: local aes128-ctr, remote aes128-ctr; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none    
DEBUG:paramiko.transport:Switch to new keys ...    
DEBUG:paramiko.transport:Trying SSH key 36f4e43a968404ef8e7f277e1429f0fd    
DEBUG:paramiko.transport:userauth is OK    
INFO:paramiko.transport:Authentication (publickey) failed.    
DEBUG:paramiko.transport:Trying discovered key 54b98c4b8ba454594e9df58bc8f9b5e7 in /home/apache/.ssh/id_rsa
DEBUG:paramiko.transport:userauth is OK    
INFO:paramiko.transport:Authentication (publickey) failed.    
DEBUG:paramiko.transport:Trying discovered key d2a34d82ebe4439672bd2c16540c5bb4 in /home/apache/.ssh/id_dsa    
DEBUG:paramiko.transport:userauth is OK    
INFO:paramiko.transport:Authentication (publickey) failed.    
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/apache/miniconda/lib/python2.7/site-packages/paramiko-1.15.2-py2.7.egg/paramiko/client.py", line 307, in connect
  File "/home/apache/miniconda/lib/python2.7/site-packages/paramiko-1.15.2-py2.7.egg/paramiko/client.py", line 519, in _auth
paramiko.ssh_exception.AuthenticationException: Authentication failed.
>>> DEBUG:paramiko.transport:EOF in transport thread

EDIT: What really puzzles me is that this works going between two actual machines on the network. I can ssh into apache@virtualmachine and from apache in the terminal. I have verified that the key is added during ftp.put(). Though I can't find anything about paramiko having issues going out form a VM.

EDIT2: Using the "look_for_keys=False' gives the same output, but only uses the given key. Note: it is using a different key as I regenerated one today different from yesterdays.

ssh.connect(server, username=username, pkey=rkey, look_for_keys=False)
DEBUG:paramiko.transport:starting thread (client mode): 0x84938990L
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_5.3)
DEBUG:paramiko.transport:kex algos:[u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1'] server key:[u'ssh-rsa', u'ssh-dss'] client encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'[email protected]'] server encrypt:[u'aes128-ctr', u'aes192-ctr', u'aes256-ctr', u'arcfour256', u'arcfour128', u'aes128-cbc', u'3des-cbc', u'blowfish-cbc', u'cast128-cbc', u'aes192-cbc', u'aes256-cbc', u'arcfour', u'[email protected]'] client mac:[u'hmac-md5', u'hmac-sha1', u'[email protected]', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'[email protected]', u'hmac-sha1-96', u'hmac-md5-96'] server mac:[u'hmac-md5', u'hmac-sha1', u'[email protected]', u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-ripemd160', u'[email protected]', u'hmac-sha1-96', u'hmac-md5-96'] client compress:[u'none', u'[email protected]'] server compress:[u'none', u'[email protected]'] client lang:[u''] server lang:[u''] kex follows?False
DEBUG:paramiko.transport:Ciphers agreed: local=aes128-ctr, remote=aes128-ctr
DEBUG:paramiko.transport:using kex diffie-hellman-group14-sha1; server key type ssh-rsa; cipher: local aes128-ctr, remote aes128-ctr; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none
DEBUG:paramiko.transport:Switch to new keys ...
DEBUG:paramiko.transport:Trying SSH key eb06556f5c3461c6e8c4fe70398717e3
DEBUG:paramiko.transport:userauth is OK
INFO:paramiko.transport:Authentication (publickey) failed.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/apache/miniconda/lib/python2.7/site-packages/paramiko-1.15.2-py2.7.egg/paramiko/client.py", line 307, in connect
  File "/home/apache/miniconda/lib/python2.7/site-packages/paramiko-1.15.2-py2.7.egg/paramiko/client.py", line 519, in _auth
paramiko.ssh_exception.AuthenticationException: Authentication failed.
>>> DEBUG:paramiko.transport:EOF in transport thread

UPDATE: I got the connect call to work going from the VM to the machine hosting the machine. Not sure where to think this narrows down the problem :/

Answer 1

It looks to me from the output you provide as if paramiko is trying multiple different keys (before getting to the right key that authorizes the login) located in the same key file. There is usually a maximum amount of attempts, which is why authentication might fail. This often happens when you have lots of keys in yout .ssh/ folder and you use ssh to log in without the -o IdentitiesOnly=yes option (this happens even if you use -i path/to/key to specify a specific key file). How that translates to paramiko I don't know, unfortunately, but I assume that library allows you to specify the key more precisely. However, I think you'll want to set look_for_keys to False for ssh.connect, and separate the different keys into different files (one file for each server?).

Answer 2

TIL permission bits on .ssh/authorized_keys matter... can only be writable by the owner.