10

Recently I want to try Z shell in Mac. But I'd like to continue also saving the command history to ~/.persistent_history, which was what I did in Bash (ref).

However, the script in the ref link doesn't work under Zsh:

log_bash_persistent_history()
{
   [[
     $(history 1) =~ ^\ *[0-9]+\ +([^\ ]+\ [^\ ]+)\ +(.*)$
   ]]
   local date_part="${BASH_REMATCH[1]}"
   local command_part="${BASH_REMATCH[2]}"
   if [ "$command_part" != "$PERSISTENT_HISTORY_LAST" ]
   then
     echo $date_part "|" "$command_part" >> ~/.persistent_history
     export PERSISTENT_HISTORY_LAST="$command_part"
   fi
}
run_on_prompt_command()
{
   log_bash_persistent_history
}
PROMPT_COMMAND="run_on_prompt_command"

Is there anyone who can help me get it working? Many thanks!

5
  • 1
    This should help with replacing PROMPT_COMMAND. Replacing the [[ usage should be doable with grep -o or cut or similar but depends on the exact output of history in zsh. May 15, 2015 at 1:24
  • @EtanReisner Thanks so much! For the PROMPT_COMMAND, the link should be helpful. For [[ part, I just found with the command history, bash will give the newest one (in this case, it's history) at the last line. But under Zsh, history command won't return the newest one, it returns the command used before history at the last line. Any ideas? :-) May 15, 2015 at 1:47
  • Well, I see no reason in reinventing the wheel. Just set HISTFILE and set HISTSIZE and SAVEHIST to some ridiculously large size (mine are 100,000, and I see no reason to make them larger, since I log all my terminal sessions anyway in iTerm2 — that's all commands + output, with time down to seconds in my prompt). The default history format has POSIX timestamps associated, which is more accurate than yours since yours don't have tzinfo.
    – 4ae1e1
    May 15, 2015 at 4:52
  • 1
    (If you want to combine zsh and bash history — well, that's asking for trouble. The syntax of both shells are incompatible in many ways, especially if you have customized your zsh to a certain extent.)
    – 4ae1e1
    May 15, 2015 at 4:54
  • @4ae1e1 Yes, you brought up a great point. I'll think about it. Thanks so much! May 15, 2015 at 23:43

4 Answers 4

6

After so much Googling, I finally found out the way to do this. First, in ~/.zshrc, add the following options for history manipulation:

setopt append_history # append rather then overwrite
setopt extended_history # save timestamp
setopt inc_append_history # add history immediately after typing a command

In short, these three options will record every input_time+command to ~/.zsh_history immediately. Then, put this function into ~/.zshrc:

precmd() { # This is a function that will be executed before every prompt
    local date_part="$(tail -1 ~/.zsh_history | cut -c 3-12)"
    local fmt_date="$(date -d @${date_part} +'%Y-%m-%d %H:%M:%S')"
    # For older version of command "date", comment the last line and uncomment the next line
    #local fmt_date="$(date -j -f '%s' ${date_part} +'%Y-%m-%d %H:%M:%S')"
    local command_part="$(tail -1 ~/.zsh_history | cut -c 16-)"
    if [ "$command_part" != "$PERSISTENT_HISTORY_LAST" ]
    then
        echo "${fmt_date} | ${command_part}"  >> ~/.persistent_history
        export PERSISTENT_HISTORY_LAST="$command_part"
    fi
}

Since I use both bash and zsh, so I want a file that can save all their history commands. In this case, I can easily search all of them using "grep".

4
  • This fails with multi line commands. The following works: local date_part="$(gawk 'substr($0, 0, 1) == ":" {print;}' ~/.zsh_history | tail -1 | cut -c 3-12)" Aug 10, 2016 at 7:13
  • 1
    append_history and inc_append_history are mutually exclusive btw. Specifying them both does nothing in addition to just setting inc_append_history. Based on my reading of Zsh's options, append_history, inc_append_history, inc_append_history_time, and share_history are all mutually exclusive. Mar 19, 2018 at 22:16
  • @KeithDevens Sorry, where exactly did you find append_history and inc_append_history are mutually exclusive in the documentation? Jun 11, 2018 at 8:01
  • 1
    @astroboylrx from the descriptions in the docs I linked. For example, the docs say inc_append_history "...works like APPEND_HISTORY except that new history lines are added to the $HISTFILE incrementally". So there's no need to specify both append_history and inc_append_history. inc_append_history_time is more explicit: "This option is only useful if INC_APPEND_HISTORY and SHARE_HISTORY are turned off. The three options should be considered mutually exclusive." Jun 11, 2018 at 20:36
2

Can't comment yet (and this went beyond a simple correction), so I'll add this as an answer.

This correction to the accepted answer doesn't quite work when, for example, the last command took quite a bit of time to execute - you'll get stray numbers and ; in your command, like this:

2017-07-22 19:02:42 | 3;micro ~/.zshrc && . ~/.zshrc

This can be fixed by replacing the sed -re '1s/.{15}//' in command_part with a slightly longer gawk, which also avoids us a pipeline:

local command_part="$(gawk "
  NR == $line_num_last {
    pivot = match(\$0, \";\");
    print substr(\$0, pivot+1);
  }
  NR > $line_num_last {
    print;
  }" ~/.zsh_history)"

It also has problems when dealing with multiline commands where one of the lines begin with :. This can be (mostly) fixed by replacing grep -ane '^:' ~/.zsh_history in line_num_last with grep -anE '^: [0-9]{10}:[0-9]*?;' ~/.zsh_history - I say mostly because a command could conceivably contain a string matching that expression. Say,

% naughty "multiline
> command
> : 0123456789:123;but a command I'm not
> "

Which will result in a clobbered record in ~/.persistent_history.

To fix this we need, in turn, to check whether the previous redord ends with \ (there might be other conditions but I'm not familiar yet with this history format), and if so try the previous match.

_get_line_num_last () {
  local attempts=0
  local line=0
  while true; do
    # Greps the last two lines that can be considered history records
    local lines="$(grep -anE '^: [0-9]{10}:[0-9]*?;' ~/.zsh_history | \
                 tail -n $((2 + attempts)) | head -2)"
    local previous_line="$(echo "$lines" | head -1)"
    # Gets the line number of the line being tested
    local line_attempt=$(echo "$lines" | tail -1 | cut -d':' -f1 | tr -d '\n')
    # If the previous (possible) history records ends with `\`, then the
    # _current_ one is part of a multiline command; try again.
    # Probably. Unless it was in turn in the middle of a multi-line
    # command. And that's why the last line should be saved.
    if [[ $line_attempt -ne $HISTORY_LAST_LINE ]] && \
       [[ $previous_line == *"\\" ]] && [[ $attempts -eq 0 ]];
    then
      ((attempts+=1))
    else
      line=$line_attempt
      break
    fi
  done
  echo "$line"
}
precmd() {
  local line_num_last="$(_get_line_num_last)"
  local date_part="$(gawk "NR == $line_num_last {print;}" ~/.zsh_history | cut -c 3-12)"
  local fmt_date="$(date -d @${date_part} +'%Y-%m-%d %H:%M:%S')"
  # I use awk itself to split the _first_ line only at the first `;`
  local command_part="$(gawk "
    NR == $line_num_last {
      pivot = match(\$0, \";\");
      print substr(\$0, pivot+1);
    }
    NR > $line_num_last {
      print;
    }" ~/.zsh_history)"
  if [ "$command_part" != "$PERSISTENT_HISTORY_LAST" ]
  then
    echo "${fmt_date} | ${command_part}" >> ~/.persistent_history
    export PERSISTENT_HISTORY_LAST="$command_part"
    export HISTORY_LAST_LINE=$((1 + $(wc -l < ~/.zsh_history)))
  fi
}
1
  • Sorry for the late acceptance. You were right about commands beginning with :. Thanks for the insightful digging. :-) Jun 11, 2018 at 7:58
2

The original answer is mostly good, but to handle multi-line commands that also contain the character ':' for example this works:

local line_num_last=$(grep -ane '^:' ~/.zsh_history | tail -1 | cut -d':' -f1 | tr -d '\n')
local date_part="$(gawk "NR == $line_num_last {print;}" ~/.zsh_history | cut -c 3-12)"
local fmt_date="$(date -d @${date_part} +'%Y-%m-%d %H:%M:%S')"
local command_part="$(gawk "NR >= $line_num_last {print;}" ~/.zsh_history | sed -re '1s/.{15}//')"
2
  • Cool! I haven't thought about multi-line commands. This is very helpful. Thank you so much! :-) Aug 10, 2016 at 20:36
  • I add another line to make the multi-line commands stored as one-line commands: local one_line_command=${(Q)command_part//\\$'\n'/}. Now it looks really great. :-) Aug 11, 2016 at 4:31
0

If you want to be able to source something that will add persistent history for both bash and zsh, try this:

# You should source this file from both .zshrc and .bashrc

if [ -n "${ZSH_VERSION}" ]; then
    setopt append_history # append rather then overwrite
    setopt extended_history # save timestamp
    setopt inc_append_history # add history immediately after typing a command

    _get_line_num_last () {
      local attempts=0
      local line=0
      while true; do
        # Greps the last two lines that can be considered history records
        local lines="$(grep -anE '^: [0-9]{10}:[0-9]*?;' ~/.zsh_history | \
                     tail -n $((2 + attempts)) | head -2)"
        local previous_line="$(echo "$lines" | head -1)"
        # Gets the line number of the line being tested
        local line_attempt=$(echo "$lines" | tail -1 | cut -d':' -f1 | tr -d '\n')
        # If the previous (possible) history records ends with `\`, then the
        # _current_ one is part of a multiline command; try again.
        # Probably. Unless it was in turn in the middle of a multi-line
        # command. And that's why the last line should be saved.
        if [[ $line_attempt -ne $HISTORY_LAST_LINE ]] && \
           [[ $previous_line == *"\\" ]] && [[ $attempts -eq 0 ]];
        then
          ((attempts+=1))
        else
          line=$line_attempt
          break
        fi
      done
      echo "$line"
    }

    precmd() {
      local line_num_last="$(_get_line_num_last)"
      local date_part="$(awk "NR == $line_num_last {print;}" ~/.zsh_history | cut -c 3-12)"
      # Try to get date with non-mac date function.
      local fmt_date="$(date -d @${date_part} +'%Y-%m-%d %H:%M:%S')" >& /dev/null
      # Try again with mac date function if that failed.
      if [ -z "$fmt_date" ]; then
          local fmt_date="$(date -r 1623959079 +'%Y-%m-%d %H:%M:%S')" >& /dev/null
      fi
      # I use awk itself to split the _first_ line only at the first `;`
      local command_part="$(awk "
        NR == $line_num_last {
          pivot = match(\$0, \";\");
          print substr(\$0, pivot+1);
        }
        NR > $line_num_last {
          print;
        }" ~/.zsh_history)"
      if [ "$command_part" != "$PERSISTENT_HISTORY_LAST" ]
      then
        echo "${fmt_date} | ${command_part}" >> ~/.persistent_history
        export PERSISTENT_HISTORY_LAST="$command_part"
        export HISTORY_LAST_LINE=$((1 + $(wc -l < ~/.zsh_history)))
      fi
    }
elif [ -n "${BASH_VERSION}" ]; then
    log_bash_persistent_history()
    {
      [[
        $(history 1) =~ ^\ *[0-9]+\ +([^\ ]+\ [^\ ]+)\ +(.*)$
      ]]
      local date_part="${BASH_REMATCH[1]}"
      local command_part="${BASH_REMATCH[2]}"
      if [ "$command_part" != "$PERSISTENT_HISTORY_LAST" ]
      then
        echo $date_part "|" "$command_part" >> ~/.persistent_history
        export PERSISTENT_HISTORY_LAST="$command_part"
      fi
    }
    export PROMPT_COMMAND="log_bash_persistent_history"
fi

export HISTSIZE=1000000
export HISTFILESIZE=-1
export HISTCONTROL=ignoredups:erasedups
export HISTTIMEFORMAT="%F %T  "

alias persistent_history='cat ~/.persistent_history'
alias ph='cat ~/.persistent_history'
alias phgrep='ph | grep'
alias phg='ph | grep'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.