Issues with ssh-agent connecting to a large number of hosts at once

Frank Sorenson frank at tuxrocks.com
Wed Apr 22 13:15:02 MDT 2009


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bob Belnap wrote:
<snip>
> root at chub:~# ls /proc/29019/fd/
<snip>

Suppose I probably should have asked for "ls -l", since the list of fd's
itself isn't too especially helpful.  Perhaps "lsof -p <pid>" would have
been even more helpful, but it looks like you've determined what the FD
refers to anyway:

<snip>
> root at chub:~# readlink  /proc/29019/fd/160
> socket:[6380248]
> 
> I believe this should map to:
> 
> bob at chub:~$ netstat -anp  | grep 6380248
> unix  3      [ ]         STREAM     CONNECTED     6380248
> -                   /tmp/keyring-gNQ6hA/ssh
<snip>

Is the ssh-agent running as a user, or as root?  Can you verify that the
user's limits aren't getting in the way (ulimit -a).  You've confirmed
with /proc/sys/fs/file-nr that you're not running into limits there?

> I have plenty of entropy available, it only goes down slightly during the
> whole process.
<snip>
Good to know.  Just wanted to make sure that wasn't an issue.

> Another clue to the puzzle.  I have 1300 or so machines in a DC in Hong
> Kong, only available through a jump server in the same DC.  If I'm running
> my agent on my local machine, through the jump server, and connect to all
> the machines, connections time out, agent locks up, etc.  However, if I copy
> my keys to the jump box, and run the agent from there, no connections fail,
> and all connections complete very quickly.  I assume that this is because
> connections open and close quickly enough that whatever limit I'm hitting
> isn't reached (netstat snapshots every second show around 200 max concurrent
> connections).

Aha.  That does sound like it may be helpful information.

When connecting through the jump server, does it create these hundreds
of simultaneous connections from your host, or a single one to the jump
server which then fans out the connections?

I would also verify that entropy is still available on the jump server
and ake sure that the jump server has appropriate settings in
/etc/ssh/sshd_config for AllowAgentForwarding, MaxSessions, and
MaxStartups (see the manpage for sshd_config).

Frank
- --
Frank Sorenson - KD7TZK
Linux Systems Engineer, DSS Engineering, UBS AG
frank at tuxrocks.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAknvbLYACgkQaI0dwg4A47ziDwCfStA1CHPTp/r5hwV4gysln/rH
ZewAoLhTvqad2y9oMRGOW+tkjA0f1AgA
=0F2W
-----END PGP SIGNATURE-----



More information about the PLUG mailing list