535

How do I edit a file in a single sed command? Currently, I have to manually stream the edited content into a new file and then rename the new file to the original file name.

I tried sed -i, but my Solaris system said that -i is an illegal option. Is there a different way?

3
  • 11
    -i is an option in gnu sed, but is not in standard sed. However, it streams the content to a new file and then renames the file so it is not what you want. Commented Oct 2, 2012 at 18:58
  • It's not possibly to satisfy all your constraints. You could use ed rather than sed, or you stream via a memory buffer using sed "$f" | sponge "$f", or you could install GNU sed. Which of those best addresses your particular situation is beyond our knowledge (for one thing, it probably depends on the size of file you are processing). Commented Nov 14, 2016 at 16:40
  • Does this answer your question? Alternative to `sed -i` on Solaris Commented Nov 8, 2023 at 9:07

15 Answers 15

876

The -i option streams the edited content into a new file and then renames it behind the scenes, anyway.

Example:

sed -i 's/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' filename

while on macOS you need:

sed -i '' 's/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g' filename
16
  • 6
    at least it does it for me so i don't have to
    – amphibient
    Commented Oct 2, 2012 at 18:47
  • 4
    example: sed -i "s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g" <file> Commented Apr 17, 2014 at 13:30
  • 4
    @maths: It does, but maybe somewhere else or for a shorter time?
    – choroba
    Commented Jul 3, 2014 at 5:38
  • 4
    on Yocto build system following example works: sed -i -e 's,STRING with SPACES,STRING to REPLACE with,g' filename - i.e. important are commas , instead of slashes / Commented Apr 29, 2018 at 21:17
  • 7
    +1 for mac, this is probably the 3rd time (over the last 4 years) I forgot that it needs an extra par of quotes and had to return to this post. But in my defense 95% of the times I use my mac to just ssh into a linux box.
    – Ali
    Commented May 1, 2020 at 13:06
91

On a system where sed does not have the ability to edit files in place, I think the better solution would be to use perl:

perl -pi -e 's/foo/bar/g' file.txt

Although this does create a temporary file, it replaces the original because an empty in place suffix/extension has been supplied.

10
  • 21
    Not only does it create a temporary file, it also breaks hard links (eg, instead of changing the contents of the file, it replaces the file with a new file of the same name.) Sometimes this is the desired behavior, sometimes it is acceptable, but when there are multiple links to a file it is almost always the wrong behavior. Commented Oct 3, 2012 at 11:37
  • 4
    @DwightSpencer: I believe that all systems without Perl are broken. A single command trumps a chain of commands when one wants elevated permissions and must use sudo. In my mind, the question is about how to avoid manually creating and renaming a temporary file without having to install the latest GNU or BSD sed. Just use Perl.
    – Steve
    Commented Feb 20, 2014 at 0:42
  • 6
    @PetrPeller: Really? If you read the question carefully, you would understand that the OP is trying to avoid something like, sed 's/foo/bar/g' file.txt > file.tmp && mv file.tmp file.txt. Just because in-place editing does the rename using a temporary file, doesn't mean that he/she must perform a manual rename when the option is not available. There are other tools out there that can do what he/she wants, and Perl is the obvious choice. How is this not an answer to the original question?
    – Steve
    Commented Aug 1, 2014 at 2:54
  • 4
    @a_arias: You're right; he did. But he can't achieve what he wants using Solaris sed. The question was actually a very good one, but it's value has been diminished by people who either can't read man pages or can't be bothered to actually read questions here on SO.
    – Steve
    Commented Oct 14, 2014 at 1:28
  • 5
    @EdwardGarson: IIRC, Solaris ships with Perl but not Python or Ruby. And like I've said before, the OP can't achieve the desired outcome using Solaris sed.
    – Steve
    Commented Jul 23, 2015 at 1:25
86

Note that on OS X you might get strange errors like "invalid command code" or other strange errors when running this command. To fix this issue try

sed -i '' -e "s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g" <file>

This is because on the OSX version of sed, the -i option expects an extension argument so your command is actually parsed as the extension argument and the file path is interpreted as the command code. Source: https://stackoverflow.com/a/19457213

1
  • 6
    This is another oddity of OS X. Fortunately, a brew install gnu-sed along with a PATH will allow you to use the GNU version of sed. Thanks for mentioning; a definite head scratcher.
    – Doug
    Commented Jul 3, 2019 at 13:28
35

The following works fine on my mac

sed -i.bak 's/foo/bar/g' sample

We are replacing foo with bar in sample file. Backup of original file will be saved in sample.bak

For editing inline without backup, use the following command

sed -i'' 's/foo/bar/g' sample
3
  • Any way how to prevent keeping backup files? Commented Aug 1, 2014 at 8:03
  • @PetrPeller, You can use the mentioned command for the same... sed -i '' 's/foo/bar/g' sample
    – minhas23
    Commented Aug 1, 2014 at 9:46
  • 1
    I like this answer "sed -i.bak" because it works with both OS X and GNU Commented Mar 3, 2021 at 19:36
22

One thing to note, sed cannot write files on its own as the sole purpose of sed is to act as an editor on the "stream" (ie pipelines of stdin, stdout, stderr, and other >&n buffers, sockets and the like). With this in mind you can use another command tee to write the output back to the file. Another option is to create a patch from piping the content into diff.

Tee method

sed '/regex/' <file> | tee <file>

Patch method

sed '/regex/' <file> | diff -p <file> /dev/stdin | patch

UPDATE:

Also, note that patch will get the file to change from line 1 of the diff output:

Patch does not need to know which file to access as this is found in the first line of the output from diff:

$ echo foobar | tee fubar

$ sed 's/oo/u/' fubar | diff -p fubar /dev/stdin
*** fubar   2014-03-15 18:06:09.000000000 -0500
--- /dev/stdin  2014-03-15 18:06:41.000000000 -0500
***************
*** 1 ****
! foobar
--- 1 ----
! fubar

$ sed 's/oo/u/' fubar | diff -p fubar /dev/stdin | patch
patching file fubar
6
  • 3
    /dev/stdin 2014-03-15, answered Jan 7 - are you a time traveller? Commented Apr 17, 2014 at 13:55
  • On Windows, using msysgit, /dev/stdin doesn't exist, so you have to replace /dev/stdin with '-', a single hyphen without the quotes, so the following commands should work: $ sed 's/oo/u/' fubar | diff -p fubar - and $ sed 's/oo/u/' fubar | diff -p fubar - | patch
    – Jim Raden
    Commented Nov 6, 2014 at 16:30
  • @AdrianFrühwirth I do have a blue police box somewhere's around and for some reason they do call me the Doctor at work. But honestly, looks like there's really bad clock drift on the system at the time. Commented Apr 28, 2016 at 16:29
  • 4
    These solutions look to me relying on particular buffering behavior. I suspect for bigger files these can break as while sed is reading, the file may start changing underneath by the patch command, or even worse by the tee command that will truncate the file. Actually I'm not sure if patch will not truncate as well. I think saving output to another file and then cat-ing into original would be safer. Commented May 26, 2016 at 12:26
  • real in-place answer from @william-pursell Commented May 26, 2016 at 13:09
17

Versions of sed that support the -i option for editing a file in place write to a temporary file and then rename the file.

Alternatively, you can just use ed. For example, to change all occurrences of foo to bar in the file file.txt, you can do:

echo ',s/foo/bar/g; w' | tr \; '\012' | ed -s file.txt

Syntax is similar to sed, but certainly not exactly the same.

Even if you don't have a -i supporting sed, you can easily write a script to do the work for you. Instead of sed -i 's/foo/bar/g' file, you could do inline file sed 's/foo/bar/g'. Such a script is trivial to write. For example:

#!/bin/sh
IN=$1
shift
trap 'rm -f "$tmp"' 0
tmp=$( mktemp )
<"$IN" "$@" >"$tmp" && cat "$tmp" > "$IN"  # preserve hard links

should be adequate for most uses.

5
  • 1
    so for example, how would you name this script and how would you call it?
    – amphibient
    Commented Oct 2, 2012 at 21:00
  • 2
    I would call it inline and invoke it as describe above: inline inputfile sed 's/foo/bar/g' Commented Oct 2, 2012 at 21:11
  • 2
    wow, ed only answer that really does in-place. First time I ever need ed. Thank you. A little optimization is to use echo -e ",s/foo/bar/g\012 w" or echo $',s/foo/bar/g\012 w' where shell allows it to avoid extra tr call. Commented May 26, 2016 at 13:08
  • 4
    ed doesn't really do the modifications in-place. From strace output, GNU ed creates a temp file, writes the changes to the temp file, then rewrites the entire original file, preserving hard links. From truss output, Solaris 11 ed also uses a temp file, but renames the temp file to the original file name upon saving with the w command, destroying any hard links. Commented Jan 19, 2018 at 11:06
  • 1
    @AndrewHenle That's discouraging. The ed that ships with current macos (Mojave, whatever that marketing jargon means) still preserves hardlinks. YMMV Commented Dec 2, 2019 at 13:49
16

You could use vi

vi -c '%s/foo/bar/g' my.txt -c 'wq'
10

sed supports in-place editing. From man sed:

-i[SUFFIX], --in-place[=SUFFIX]

    edit files in place (makes backup if extension supplied)

Example:

Let's say you have a file hello.txtwith the text:

hello world!

If you want to keep a backup of the old file, use:

sed -i.bak 's/hello/bonjour' hello.txt

You will end up with two files: hello.txt with the content:

bonjour world!

and hello.txt.bak with the old content.

If you don't want to keep a copy, just don't pass the extension parameter.

1
  • 5
    The OP specifically mentions that his platform does not support this (popular but) nonstandard option.
    – tripleee
    Commented Jan 18, 2016 at 20:08
5

If you are replacing the same amount of characters and after carefully reading “In-place” editing of files...

You can also use the redirection operator <> to open the file to read and write:

sed 's/foo/bar/g' file 1<> file

See it live:

$ cat file
hello
i am here                           # see "here"
$ sed 's/here/away/' file 1<> file  # Run the `sed` command
$ cat file
hello
i am away                           # this line is changed now

From Bash Reference Manual → 3.6.10 Opening File Descriptors for Reading and Writing:

The redirection operator

[n]<>word

causes the file whose name is the expansion of word to be opened for both reading and writing on file descriptor n, or on file descriptor 0 if n is not specified. If the file does not exist, it is created.

6
  • This corrupted by file. I am not sure of the reason behind it. paste.fedoraproject.org/462017 Commented Oct 27, 2016 at 19:09
  • See lines [43-44], [88-89], [135-136] Commented Oct 27, 2016 at 19:15
  • 3
    This blogpost explains why this answer should not be used. backreference.org/2011/01/29/in-place-editing-of-files Commented Oct 27, 2016 at 19:22
  • @NehalJWani I will have a long read to the article, thanks for the heads-up! What I did not mention in the answer is that this works if it changes the same amount of characters.
    – fedorqui
    Commented Oct 28, 2016 at 6:51
  • @NehalJWani this being said, I am sorry if my answer caused harm on your files. I will update it with corrections (or delete it if necessary) after reading the links you provided.
    – fedorqui
    Commented Oct 28, 2016 at 13:04
4

Like Moneypenny said in Skyfall: "Sometimes the old ways are best." Kincade said something similar later on.

$ printf ',s/false/true/g\nw\n' | ed {YourFileHere}

Happy editing in place. Added '\nw\n' to write the file. Apologies for delay answering request.

3
  • Can you explain what this does? Commented Jan 16, 2018 at 7:53
  • 2
    It invokes ed(1), the text editor with some characters written to stdin. As someone under the age of 30, I think it's okay for me to say that ed(1) is to dinosaurs as dinosaurs are to us. Anyway, instead of opening ed and typing that stuff in, jlettvin is piping the characters in using |. @MattMontag Commented Aug 14, 2018 at 16:13
  • If you're asking what the commands do, there's two of them, terminated by a \n. [range]s/<old>/<new>/<flag> is the form of the substitute command - a paradigm still seen in many other text editors (okay, vim, a direct descendent of ed) Anyway, the g flag stands for "global", and means "Every instance on each line that you look at". The range 3,5 looks at lines 3-5. ,5 looks at StartOfFile-5, 3, looks at lines 3-EndOfFile and , looks at StartOfFile-EndOfFile. A global substitue command on every line that replaces "false" with "true". Then, the write command, w, is entered. Commented Aug 14, 2018 at 16:17
4
cp file.txt file.tmp && sed 's/foo/bar/g' < file.tmp > file.txt

Should preserve all hardlinks, since output is directed back to overwrite the contents of the original file, and avoids any need for a special version of sed.

2
  • 2
    this only makes sense if the first command was intended to be cp rather than mv
    – jhnc
    Commented Sep 10, 2023 at 16:26
  • 1
    yeah, using mv does nothing of what JJM said: it doesn't overwrite the original file, it doesn't preserve hardlinks. I changed the command to cp as mentioned by jhnc to reflect what JJM said the command would do.
    – msb
    Commented Jan 4 at 1:13
3

You didn't specify what shell you are using, but with zsh you could use the =( ) construct to achieve this. Something along the lines of:

cp =(sed ... file; sync) file

=( ) is similar to >( ) but creates a temporary file which is automatically deleted when cp terminates.

0
3

To resolve this issue on Mac I had to add some unix functions to core-utils following this.

brew install grep
==> Caveats
All commands have been installed with the prefix "g".
If you need to use these commands with their normal names, you
can add a "gnubin" directory to your PATH from your bashrc like:
  PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"

Call with gsed instead of sed. The mac default doesn't like how grep -rl displays file names with the ./ preprended.

~/my-dir/configs$ grep -rl Promise . | xargs sed -i 's/Promise/Bluebird/g'

sed: 1: "./test_config.js": invalid command code .

I also had to use xargs -I{} sed -i 's/Promise/Bluebird/g' {} for files with a space in the name.

1

Very good examples. I had the challenge to edit in place many files and the -i option seems to be the only reasonable solution using it within the find command. Here the script to add "version:" in front of the first line of each file:

find . -name pkg.json -print -exec sed -i '.bak' '1 s/^/version /' {} \;
1

In case you want to replace stings contain /,you can use ?. i.e. replace /usr/local/bin/python with /usr/bin/python3 for all *.py files.

find . -name \\*.py -exec sed -i 's?/usr/local/bin/python?/usr/bin/python3?g' {} \\;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.