93

I am trying to find a nice way to restore the SSH agent when I reconnect a disconnected tmux session.

The cause seems to be that the SSH agent session changes but the environment variable from the tmux session is not updated.

How can I automate this, before attaching the session itself? Because the session I am attaching to does not always have a bash prompt, so I cannot afford to type something inside it. It has to be something to run before creating or attaching the tmux session.

An example of the code I'm running is at https://gist.github.com/ssbarnea/8646491 -- a small ssh wrapper that is using tmux to create persistem ssh connections. This works quite well, but sometimes the ssh agent stops working so I am no longer able to use it to connect to other hosts.

1
  • 1
    You probably should mark pymkin's response as the answer. Dec 29, 2014 at 18:52

14 Answers 14

86

There's an excellent gist by Martijn Vermaat, which addresses your problem in great depth, although it is intended for screen users, so I'm adjusting it for tmux here.

To summarize:

  1. create ~/.ssh/rc if it doesn't exist yet, and add the following content:

    #!/bin/bash
    
    # Fix SSH auth socket location so agent forwarding works with tmux.
    if test "$SSH_AUTH_SOCK" ; then
      ln -sf $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock
    fi
    
  2. Make it work in tmux, add this to your ~/.tmux.conf:

    # fix ssh agent when tmux is detached
    setenv -g SSH_AUTH_SOCK $HOME/.ssh/ssh_auth_sock
    

Extra work is required if you want to enable X11 forwarding, see the gist.

14
  • 1
    When reconnecting to tmux, it doesn't update to the newest ssh_auth_sock... it stays with the one that originally created the session, so this doesn't work. Any ideas?
    – Bret
    Feb 22, 2015 at 0:35
  • 6
    I had to add a set -g update-environment -r to the .tmux in order to get this process to work. I also added some namespacing to the socket link creation: gist.github.com/bcomnes/e756624dc1d126ba2eb6
    – Bret
    Feb 22, 2015 at 1:55
  • 1
    Fixed an error with the previous GIST: had change $(hostname) to $HOSTNAME in the .screenrc and .tmux.conf files
    – Bret
    Feb 22, 2015 at 20:08
  • 2
    @Bret Your solution works fine if you detach from a tmux session, and then close SSH connection. However, it does not work if you close the ssh connection forcefully, still being attached to a tmux session (e.g. when you suddenly loose network connection and close SSH connection with [Shift ~][Enter]. Any ideas what could fix this? Apr 19, 2017 at 9:03
  • 2
    No, I gave up sorry.
    – Bret
    Jun 21, 2018 at 16:49
51

While tmux updates SSH variables by default, there is no need to

  • change/add socket path
  • change the SSH_AUTH_SOCKET variable

I like the solution by Chris Down which I changed to add function

fixssh() {
    eval $(tmux show-env    \
        |sed -n 's/^\(SSH_[^=]*\)=\(.*\)/export \1="\2"/p')
}

into ~/.bashrc. Call fixssh after attaching session or before ssh/scp/rsync.

Newer versions of tmux support -s option for show-env, so only

eval $(tmux show-env -s |grep '^SSH_')

is possible.

5
  • Thanks a lot, this is what worked for me! :) However, one small problem: when doing this in a reattached tmux pane, it doesn't work. I need to close the old pane, and open a new one. A fix for this is to detach tmux, show the $DISPLAY variable, reattach tmux, and set $DISPLAY correctly, in my case export DISPLAY=localhost:14.0, but the number seems to change at each ssh session. Sep 30, 2016 at 12:07
  • "While tmux updates SSH variables by default" What are you talking about? (I suspect that the anchor in your link is now nonexistent. Jun 26, 2018 at 19:38
  • 1
    @BrunoBronosky I fixed his link. You were right: anchor no longer existed. Looks to have a stable form now.
    – overthink
    Aug 21, 2018 at 14:04
  • 1
    This should be the top answer. It is unnecessary to create a new temporary file.
    – sjy
    Mar 7, 2019 at 8:53
  • While this solution is cleaner in that it doesn't require a temp file, it does depend on somehow executing the hook after reattaching (manually?) which conflicts with an explicit spec in the OP.
    – q.undertow
    May 26, 2022 at 22:35
28

Here's what I use for updating SSH_AUTH_SOCK inside a tmux window (based on Hans Ginzel's script):

alias fixssh='eval $(tmux showenv -s SSH_AUTH_SOCK)'

Or for tmux that does not have showenv -s:

alias fixssh='export $(tmux showenv SSH_AUTH_SOCK)'
1
  • I have added a test if SSH_AUTH_SOCK is set: alias fixssh='[[ "tmux show-env SSH_AUTH_SOCK" == "-SSH_AUTH_SOCK" ]] && echo "Warning: No SSH_AUTH_SOCK set." || export $(tmux show-env SSH_AUTH_SOCK)' Oct 25, 2022 at 7:56
6

Here is my solution which includes both approaches, and does not require extra typing when I reconnect to tmux session

alias ssh='[ -n "$TMUX" ] && eval $(tmux showenv -s SSH_AUTH_SOCK); /usr/bin/ssh'
3
  • 1
    It looks like it will execute eval $(tmux showenv -s SSH_AUTH_SOCK) in local, not remote!
    – acgtyrant
    Jun 25, 2018 at 6:56
  • I wrote this on the remote (tmux is running) and it worked as I expected. Thanks!
    – riywo
    Jan 24, 2019 at 19:00
  • This will not work when ssh is called indirectly, e.g. git fetch. Aug 30, 2022 at 9:01
6

There are lots of good answers here. But there are cases where tmux show-environment doesn't see SSH_AUTH_SOCK. In that case you can use find to locate it explicitly.

export SSH_AUTH_SOCK=$(find /tmp -path '*/ssh-*' -name 'agent*' -uid $(id -u) 2>/dev/null | tail -n1)

That's long and complicated, so I'll break it down...

01  export SSH_AUTH_SOCK=$(
02    find /tmp \
03      -path '*/ssh-*'
04      -name 'agent*'
05      -uid $(id -u)
06      2>/dev/null
07    | tail -n1
08  )
  1. export the SSH_AUTH_SOCK environment variable set to the output of the $() command substitution
  2. find files starting in /tmp
  3. limit results to only those with /ssh- in the path
  4. limit results to only those whose name begins with agent
  5. limit results to only those with a user id matching the current user
  6. silence all (permissions, etc.) errors
  7. take only the last result if there are multiple

You may be able to leave off 6 & 7 if you know that there will only be 1 result and you don't care about stderr garbage.

2
  • 2
    Out of all solutions listed here, only this worked on mac.
    – SilentGuy
    Feb 25, 2019 at 17:47
  • Nice but it may be required to take only LAST ssh agent socket. To do that, we may add sorting into find: find /tmp -path '*/ssh-*' -name 'agent*' -uid (id -u) -printf "%T+\t%p\n" 2>/dev/null | sort | tail -n1 | awk -F'\t' '{print $2 }'
    – ColCh
    Feb 28, 2020 at 14:08
4

I use a variation of the previous answers:

eval "export $(tmux show-environment -g SSH_AUTH_SOCK)"

assuming that you did the ssh agent started from the outer environment. Same goes for other environment variables such as DISPLAY.

1
  • Are you sure this works for you? With -g I get the stale value of $SSH_AUTH_SOCK. Leaving away -g does the trick. See also other answers.
    – feklee
    Aug 9, 2019 at 8:01
3

I prefer to avoid configuring TMUX (etc) and keep everything purely in ~/.ssh/. On the remote system:

Create ~/.ssh/rc:

#!/bin/bash

# Fix SSH auth socket location so agent forwarding works within tmux
if test "$SSH_AUTH_SOCK" ; then
  ln -sf $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock
fi

Add following to ~/.ssh/config so it no longer relies on $SSH_AUTH_SOCK, which goes stale in detached terminals:

Host *
  IdentityAgent ~/.ssh/ssh_auth_sock

Known limitations

  • ssh-add doesn't use ~/.ssh/config and so cannot communicate with ssh-agent. Commands like ssh-add -l produce errors, even though ssh user@host works fine, as does updating git remotes which are accessed via SSH.
2

I may have worked out a solution that is fully encapsulated in the ~/.tmux.conf configuration file. It is a different approach than modifying the ~/.bash_profile and ~/.ssh/rc.

Solution only using ~/.tmux.conf

Just cut and paste the following code into your ~/.tmux.conf

# ~/.tmux.conf

# SSH agent forwarding
#
# Ensure that SSH-Agent forwarding will work when re-attaching to the tmux
#   session from a different SSH connection (after a dropped connection).
#   This code will run upon tmux create, tmux attach, or config reload.
#
# If there is an SSH_AUTH_SOCK originally defined:
#   1) Remove all SSH related env var names from update-environment.
#      Without this, setenv cannot override variables such as SSH_AUTH_SOCK.
#      Verify update-environment with: tmux show-option -g update-environment
#   2) Force-set SSH_AUTH_SOCK to be a known location
#      /tmp/ssh_auth_sock_tmux
#   3) Force-create a link of the first found ssh-agent socket at the known location
if-shell '[ -n $SSH_AUTH_SOCK ]' " \
  set-option -sg update-environment \"DISPLAY WINDOWID XAUTHORITY\"; \
  setenv -g SSH_AUTH_SOCK /tmp/ssh_auth_sock_tmux; \
  run-shell \"ln -sf $(find /tmp/ssh-* -type s -readable | head -n 1) /tmp/ssh_auth_sock_tmux\" \
"

Caveat

The above solution along with the other solutions are susceptible to a race condition when initiating multiple connections to the same machine. Consider this:

  • Client 1 Connect: SSH to machineX, start/attach tmux (writes ssh_auth_sock link)
  • Client 2 Connect: SSH to machineX, start/attach tmux (overwrites ssh_auth_sock link)
  • Client 2 Disconnect: Client 1 is left with a stale ssh_auth_sock link, thus breaking ssh-agent

However, this solution is slightly more resilient because it only overwrites the ssh_auth_sock link upon tmux start/attach, instead of upon initialization of a bash shell ~/.bash_profile or ssh connection ~/.ssh/rc

To cover this last race condition, one may add a key binding to reload the tmux configuration with a (Ctrl-b r) key sequence.

# ~/.tmux.conf

# reload config file
bind r source-file ~/.tmux.conf

From within an active tmux session, executing this sequence when the ssh_auth_sock link goes stale will refresh the ssh-agent connection.

2

Here's a new fix to an old problem: I think it's simpler than the other fixes and there's no need to make a static socket or mess with the shell prompt or make a separate command you have to remember to run.

I added this code added to my .bashrc file:

if [[ -n $TMUX ]]; then
  _fix_ssh_agent_in_tmux () { if [[ ! -S $SSH_AUTH_SOCK ]]; then eval export $(tmux show-env | grep SSH_AUTH_SOCK); fi }
  ssh ()   { _fix_ssh_agent_in_tmux; command ssh $@; }
  scp ()   { _fix_ssh_agent_in_tmux; command scp $@; }
  git ()   { _fix_ssh_agent_in_tmux; command git $@; }
  rsync () { _fix_ssh_agent_in_tmux; command rsync $@; }
fi

If the shell is running within tmux, it redefines 'ssh' and its ilk to bash functions which test and fix SSH_AUTH_SOCK before actually running the real commands.

Note that tmux show-env -g also returns a value for SSH_AUTH_SOCK but that one is stale, I assume it's from whenever the tmux server started. The command above queries the current tmux session's environment which seems to be correct.

I'm using tmux 2.6 (ships with with Ubuntu 18.04) and it seems to work well.

1

In case other fish shell users are wondering how to deal with this when using fish (as well as for my future self!). In my fish_prompt I added a call to the following function:

function _update_tmux_ssh
  if set -q TMUX
    eval (tmux show-environment SSH_AUTH_SOCK | sed 's/\=/ /' | sed 's/^/set /')
  end
end

I suppose that more advanced *nix users would know how to replace sed with something better, but this works (tmux 3.0, fish 3.1).

1

Following up on @pymkin's answer above, add the following, which worked with tmux 3.2a on macOS 11.5.3:

  1. To ~/.tmux.conf:
# first, unset update-environment[SSH_AUTH_SOCK] (idx 3), to prevent
# the client overriding the global value
set-option -g -u update-environment[3]
# And set the global value to our static symlink'd path:
set-environment -g SSH_AUTH_SOCK $HOME/.ssh/ssh_auth_sock
  1. To ~/.ssh/rc:
#!/bin/sh
# On SSH connection, create stable auth socket path for Tmux usage
if test "$SSH_AUTH_SOCK"; then
    ln -sf "$SSH_AUTH_SOCK" ~/.ssh/ssh_auth_sock
fi

What's going on? Tmux has the semi-helpful update-environment variable/feature to pick up certain environment variables when a client connects. I.e. when you do tmux new or tmux attach, it'll update the tmux environment from when you ran those commands. That's nice for new shells or commands you run inside tmux afterwards, but it doesn't help those shells you've started prior to the latest attach. To solve this, you could use some of the other answers here to have existing shells pick up this updated environment, but that's not the route I chose.

Instead, we're setting a static value for SSH_AUTH_SOCK inside tmux, which will be ~/.ssh/ssh_auth_sock. All shells inside tmux would pick that up, and never have to be updated later. Then, we configure ssh so that, upon connection, it updates that static path with a symlink to the latest real socket that ssh knows.

The missing piece from @pymkin's answer is that Tmux will have the session value override the global value, so doing set-environment -g isn't sufficient; it gets squashed whenever you re-attach. You also have to also tell tmux not to update SSH_AUTH_SOCK in the session environment, so that the global value can make it through. That's what the set-option -g -u is about.

3
  • It's unfortunate that I have to unset SSH_AUTH_SOCK in the update-environment variable by referring to it by its index as update-environment[3], but there seems to be no way of referring to an array element by value. In any case, SSH_AUTH_SOCK is part of the default set of values in update-environment at index 3, so it's reasonably static. Dec 14, 2021 at 1:03
  • See Array options in the tmux wiki. Sadly, array options aren't explained in the tmux manpage Dec 14, 2021 at 1:19
  • Thanks! It was a real pain to get this working until I found this anwser. Unfortunately, this breaks when SSHing to another host inside my tmux session. That can be fixed by linking to ~/.ssh/ssh_auth_sock.$HOSTNAME (and setting SSH_AUTH_SOCK accordingly), but HOSTNAME is not available in .tmux.conf. I realized it's simpler to set SSH_AUTH_SOCK in my default shell's rc file (e.g. .bashrc) instead. Sep 8, 2022 at 12:12
0

After coming across so many suggestions, I finally figured out a solution that enables TMUX update the stale ssh agent after being attached. Basically, both the zshrc files on the local and remote machines need to be modified.

Insert the following codes into the local zshrc, which is based on this reference.

export SSH_AUTH_SOCK=~/.ssh/ssh-agent.$(hostname).sock
ssh-add -l 2>/dev/null >/dev/null
# The error of executing ssh-add command denotes a valid agent does not
# exist. 
if [ $? -ge 1 ]; then
  # remove the socket if it exists
  if [ -S "${SSH_AUTH_SOCK}" ]; then
    rm "${SSH_AUTH_SOCK}"
  fi
  ssh-agent -a "${SSH_AUTH_SOCK}" >/dev/null
  # one week life time
  ssh-add -t 1W path-to-private-rsa-file
fi

Insert the following code into the remote zshrc, where the tmux session will be attached.

alias fixssh='eval $(tmux showenv -s SSH_AUTH_SOCK)'

Then ssh into the remote machine. The -A option is necessary.

ssh -A username@hostname

Attach the TMUX session. Check the TMUX evironment variables

# run this command in the shell
tmux showenv -s
# or run this command after prefix CTRL+A or CTRL+B
:show-environment

Run fixssh in the previously existed panes to update the ssh agent. If a new pane is created, it will automatically get the new ssh-agent.

0

Here's another simple Bash solution, using PROMPT_COMMAND to update the SSH_* vars inside tmux before each prompt is generated. The downside to this solution is that it doesn't take effect in existing shells until a new prompt is generated, because PROMPT_COMMAND is only run before creating new prompts.

Just add this to your ~/.bashrc:

update_tmux_env () {
    # Only run for shells inside a tmux session.
    if [[ -n "$TMUX" ]]; then
        eval $(tmux show-env -s | grep '^SSH_')
    fi
}
export PROMPT_COMMAND=update_tmux_env
0

This is my small practical demo code. github repo

My workaround still involves using a symbolic link to address the issue of SSH_AUTH_SOCK becoming invalid. However, I utilize a double symbolic link to differentiate between sessions and clients. The benefit is that this automated script can handle many edge cases. The downside is that this mechanism will retains few link files that have not invalid yet at the time of housekeeping check (you could check the timing in my tmux hook scripts). However, it is not a significant issue for me.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.