Issues with ssh-agent connecting to a large number of hosts at once
frank at tuxrocks.com
Wed Apr 22 13:15:02 MDT 2009
-----BEGIN PGP SIGNED MESSAGE-----
Bob Belnap wrote:
> root at chub:~# ls /proc/29019/fd/
Suppose I probably should have asked for "ls -l", since the list of fd's
itself isn't too especially helpful. Perhaps "lsof -p <pid>" would have
been even more helpful, but it looks like you've determined what the FD
refers to anyway:
> root at chub:~# readlink /proc/29019/fd/160
> I believe this should map to:
> bob at chub:~$ netstat -anp | grep 6380248
> unix 3 [ ] STREAM CONNECTED 6380248
> - /tmp/keyring-gNQ6hA/ssh
Is the ssh-agent running as a user, or as root? Can you verify that the
user's limits aren't getting in the way (ulimit -a). You've confirmed
with /proc/sys/fs/file-nr that you're not running into limits there?
> I have plenty of entropy available, it only goes down slightly during the
> whole process.
Good to know. Just wanted to make sure that wasn't an issue.
> Another clue to the puzzle. I have 1300 or so machines in a DC in Hong
> Kong, only available through a jump server in the same DC. If I'm running
> my agent on my local machine, through the jump server, and connect to all
> the machines, connections time out, agent locks up, etc. However, if I copy
> my keys to the jump box, and run the agent from there, no connections fail,
> and all connections complete very quickly. I assume that this is because
> connections open and close quickly enough that whatever limit I'm hitting
> isn't reached (netstat snapshots every second show around 200 max concurrent
Aha. That does sound like it may be helpful information.
When connecting through the jump server, does it create these hundreds
of simultaneous connections from your host, or a single one to the jump
server which then fans out the connections?
I would also verify that entropy is still available on the jump server
and ake sure that the jump server has appropriate settings in
/etc/ssh/sshd_config for AllowAgentForwarding, MaxSessions, and
MaxStartups (see the manpage for sshd_config).
Frank Sorenson - KD7TZK
Linux Systems Engineer, DSS Engineering, UBS AG
frank at tuxrocks.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
-----END PGP SIGNATURE-----
More information about the PLUG