142

This is probably a complex solution.

I am looking for a simple operator like ">>", but for prepending.

I am afraid it does not exist. I'll have to do something like

 mv myfile tmp
 cat myheader tmp > myfile

Anything smarter?

2

35 Answers 35

110

This still uses a temp file, but at least it is on one line:

echo "text" | cat - yourfile > /tmp/out && mv /tmp/out yourfile

Credit: BASH: Prepend A Text / Lines To a File

7
  • 7
    What's doing - after cat?
    – makbol
    Apr 24, 2015 at 8:56
  • Is there a way to add "text" without a new line after?
    – chishaku
    Oct 28, 2015 at 15:41
  • 4
    A warning: if yourfile is a symlink this will not do what you want.
    – dshepherd
    May 10, 2017 at 15:05
  • 3
    @makboi the - after cat is a convention used in Shell programming to mean that the input of the command should be read from stdin. The output of commands such as echo in the answer is written to the stdout, and written into stdin for the cat command using the pipe |. Think of piping as passing stdout of one command to the stdin of another, hence the name - the left hand command's output flows into the right hand command's input
    – Nick Bull
    Jan 10, 2020 at 15:28
  • 1
    Sorry to be a pedant, but && means this is as much of a "one-liner" as minifiying Javascript files creates a "one-liner", and you might confuse newbies who don't know this operator separates two commands. It'd be clearer written as the separated lines echo "text" | cat - yourfile > /tmp/out mv /tmp/out yourfile - maybe sed -i '1s/^/prepend me\n/' filename would qualify a little better for a true "one line" prepender
    – Nick Bull
    Jan 10, 2020 at 15:33
35
echo '0a
your text here
.
w' | ed some_file

ed is the Standard Editor! http://www.gnu.org/fun/jokes/ed.msg.html

3
  • 8
    or, instead of hardcoding the text to be inserted: 0r header.file Apr 8, 2011 at 10:57
  • Since the question asked for a one liner: echo -e '0a\nyour text here\n.\nw' | ed some_file
    – raphinesse
    Nov 20, 2019 at 10:56
  • great answer! should be on top IMO Mar 9, 2022 at 15:25
31

The hack below was a quick off-the-cuff answer which worked and received lots of upvotes. Then, as the question became more popular and more time passed, people started reporting that it sorta worked but weird things could happen, or it just didn't work at all. Such fun.

I recommend the 'sponge' solution posted by user222 as Sponge is part of 'moreutils' and probably on your system by default. (echo 'foo' && cat yourfile) | sponge yourfile

The solution below exploits the exact implementation of file descriptors on your system and, because implementation varies significantly between nixes, it's success is entirely system dependent, definitively non-portable, and should not be relied upon for anything even vaguely important. Sponge uses the /tmp filesystem but condenses the task to a single command.

Now, with all that out of the way the original answer was:


Creating another file descriptor for the file (exec 3<> yourfile) thence writing to that (>&3) seems to overcome the read/write on same file dilemma. Works for me on 600K files with awk. However trying the same trick using 'cat' fails.

Passing the prependage as a variable to awk (-v TEXT="$text") overcomes the literal quotes problem which prevents doing this trick with 'sed'.

#!/bin/bash
text="Hello world
What's up?"

exec 3<> yourfile && awk -v TEXT="$text" 'BEGIN {print TEXT}{print}' yourfile >&3
5
  • 3
    WARNING: check the output to make sure nothing has changed but the first line. I used this solution, and although it seemed to work, I noticed that some new lines seemed to be added. I don't understand where these came from, so I cannot be sure that this solution really has a problem, but beware.
    – conradlee
    Mar 18, 2010 at 15:58
  • 1
    Note in the bash example above that the double quotes span over the carriage return in order to demonstrate prepending multiple lines. Check your shell and text editor are being cooperative. \r\n comes to mind.
    – John Mee
    Mar 19, 2010 at 5:05
  • 4
    Use this with caution! See the following comment for why: stackoverflow.com/questions/54365/…
    – Alex
    Dec 30, 2010 at 13:45
  • 3
    If it's now unreliable, how about updating your answer with a better solution?
    – zakdances
    Oct 25, 2013 at 2:50
  • A hack is useful if and only if it is known precisely why it works and, therefore, under what carefully observed constraints. Your disclaimer notwithstanding, your hack doesn't meet these criteria ("entirely system dependent", "seems to overcome", "works for me"). If we take your disclaimer seriously, no one should use your hack - and I agree. So what are we left with? A noisy distraction. I see two options: (a) delete your answer, or (b) turn it into a cautionary tale that explains why - tempting as it may be - this approach cannot work generally.
    – mklement0
    May 17, 2017 at 1:43
20

John Mee: your method is not guaranteed to work, and will probably fail if you prepend more than 4096 byte of stuff (at least that's what happens with gnu awk, but I suppose other implementations will have similar constraints). Not only will it fail in that case, but it will enter an endless loop where it will read its own output, thereby making the file grow until all the available space is filled.

Try it for yourself:

exec 3<>myfile && awk 'BEGIN{for(i=1;i<=1100;i++)print i}{print}' myfile >&3

(warning: kill it after a while or it will fill the filesystem)

Moreover, it's very dangerous to edit files that way, and it's very bad advice, as if something happens while the file is being edited (crash, disk full) you're almost guaranteed to be left with the file in an inconsistent state.

2
  • 6
    This should probably be a comment, not a separate answer.
    – lkraav
    Aug 13, 2011 at 9:50
  • 11
    @Ikraav: maybe, but the "comment" is very long and elaborate (and useful), it will not look good and perhaps doesn't fit with StackOverflow's limited comment capacity anyway. Oct 17, 2011 at 9:45
20

If you need this on computers you control, install the package "moreutils" and use "sponge". Then you can do:

cat header myfile | sponge myfile
1
  • 2
    This worked nicely for me when combined with @Vinko Vrsalovic's answer: { echo "prepended text"; cat myfile } | sponge myfile Jan 24, 2020 at 17:32
19

Not possible without a temp file, but here goes a oneliner

{ echo foo; cat oldfile; } > newfile && mv newfile oldfile

You can use other tools such as ed or perl to do it without temp files.

1
  • Another link destroyer. May 1, 2022 at 23:30
16

It may be worth noting that it often is a good idea to safely generate the temporary file using a utility like mktemp, at least if the script will ever be executed with root privileges. You could for example do the following (again in bash):

(tmpfile=`mktemp` && { echo "prepended text" | cat - yourfile > $tmpfile && mv $tmpfile yourfile; } )
13

Using a bash heredoc you can avoid the need for a tmp file:

cat <<-EOF > myfile
  $(echo this is prepended)
  $(cat myfile)
EOF

This works because $(cat myfile) is evaluated when the bash script is evaluated, before the cat with redirect is executed.

10

assuming that the file you want to edit is my.txt

$cat my.txt    
this is the regular file

And the file you want to prepend is header

$ cat header
this is the header

Be sure to have a final blank line in the header file.
Now you can prepend it with

$cat header <(cat my.txt) > my.txt

You end up with

$ cat my.txt
this is the header
this is the regular file

As far as I know this only works in 'bash'.

5
  • why does this work? Do the parentheses cause the middle cat to execute in a separate shell and dump the entire file at once?
    – Catskul
    Oct 26, 2010 at 19:06
  • I think the <(cat my.txt) creates a temporary file somewhere in the file system. This file is then read before the original file is modified.
    – cb0
    Nov 9, 2010 at 15:08
  • Someone once showed me what you can do with the "<( )" syntax. However I've never learned how this methode is called neither could I find something in the bash manpage. If someone knows more about it please let me know.
    – cb0
    Nov 9, 2010 at 15:11
  • 1
    Didn't work for me. Don't know why. I'm using GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin14), I'm on OS X Yosemite. I ended up with two lines of this is the header in my.txt. Even after I updated Bash to 4.3.42(1)-release I get the same result. Nov 19, 2015 at 16:51
  • 1
    Bash doesn't create a temporary file, it creates a FIFO (pipe) to which the command in the process substitution (<(...)) writes, so there's no guarantee that my.txt is read in full up front, without which this technique won't work.
    – mklement0
    May 15, 2017 at 4:42
9

When you start trying to do things that become difficult in shell-script, I would strongly suggest looking into rewriting the script in a "proper" scripting language (Python/Perl/Ruby/etc)

As for prepending a line to a file, it's not possible to do this via piping, as when you do anything like cat blah.txt | grep something > blah.txt, it inadvertently blanks the file. There is a small utility command called sponge you can install (you do cat blah.txt | grep something | sponge blah.txt and it buffers the contents of the file, then writes it to the file). It is similar to a temp file but you dont have to do that explicitly. but I would say that's a "worse" requirement than, say, Perl.

There may be a way to do it via awk, or similar, but if you have to use shell-script, I think a temp file is by far the easiest(/only?) way..

0
9

Like Daniel Velkov suggests, use tee.
To me, that's simple smart solution:

{ echo foo; cat bar; } | tee bar > /dev/null
1
  • Don't try to be clever to optimize this to { echo foo; cat bar; } > bar. I don't know why, but this will be a recursive loop hole. May 1, 2022 at 23:36
7

EDIT: This is broken. See Weird behavior when prepending to a file with cat and tee

The workaround to the overwrite problem is using tee:

cat header main | tee main > /dev/null
3
  • Hung for a while. Ended with "tee: main: No space left on device". main was several GBs, although both original text files were just a few KB.
    – Marcos
    Aug 14, 2014 at 9:10
  • 4
    If the solution you posted is broken (and in fact fills up users harddrives), do the community a favor and delete your answer.
    – aioobe
    Sep 26, 2014 at 9:06
  • 1
    Can you point me to a guideline which says that wrong answers should be deleted?
    – Daniel
    Sep 26, 2014 at 17:51
6
sed -i -e "1s/^/new first line\n/" old_file.txt
1
  • exactly what I was looking for, but indeed not the right answer to the question
    – masterxilo
    Mar 19, 2018 at 17:03
3

The one which I use. This one allows you to specify order, extra chars, etc in the way you like it:

echo -e "TEXTFIRSt\n$(< header)\n$(< my.txt)" > my.txt

P.S: only it's not working if files contains text with backslash, cause it gets interpreted as escape characters

3

Mostly for fun/shell golf, but

ex -c '0r myheader|x' myfile

will do the trick, and there are no pipelines or redirections. Of course, vi/ex isn't really for noninteractive use, so vi will flash up briefly.

1
  • Best recipe from my point of view as it uses an incumbent command and does that in a straightforward way.
    – Diego
    May 25, 2016 at 19:36
3

With $( command ) you can write the output of a command into a variable. So I did it in three commands in one line and no temp file.

originalContent=$(cat targetfile) && echo "text to prepend" > targetfile && echo "$originalContent" >> targetfile
1
  • But instead of temp file, your content exists in memory in your bash process. This may or may not be a problem. May 2, 2022 at 0:28
2

Why not simply use the ed command (as already suggested by fluffle here)?

ed reads the whole file into memory and automatically performs an in-place file edit!

So, if your file is not that huge ...

# cf. "Editing files with the ed text editor from scripts.",
# http://wiki.bash-hackers.org/doku.php?id=howto:edit-ed

prepend() {
   printf '%s\n' H 1i "${1}" . wq | ed -s "${2}"
}

echo 'Hello, world!' > myfile
prepend 'line to prepend' myfile

Yet another workaround would be using open file handles as suggested by Jürgen Hötzel in Redirect output from sed 's/c/d/' myFile to myFile

echo cat > manipulate.txt
exec 3<manipulate.txt
# Prevent open file from being truncated:
rm manipulate.txt
sed 's/cat/dog/' <&3 > manipulate.txt

All this could be put on a single line, of course.

2

A variant on cb0's solution for "no temp file" to prepend fixed text:

echo "text to prepend" | cat - file_to_be_modified | ( cat > file_to_be_modified ) 

Again this relies on sub-shell execution - the (..) - to avoid the cat refusing to have the same file for input and output.

Note: Liked this solution. However, in my Mac the original file is lost (thought it shouldn't but it does). That could be fixed by writing your solution as: echo "text to prepend" | cat - file_to_be_modified | cat > tmp_file; mv tmp_file file_to_be_modified

1
  • 1
    I too find the original file lost. Ubuntu Lucid.
    – Hedgehog
    Feb 6, 2012 at 3:36
2

Here's what I discovered:

echo -e "header \n$(cat file)" >file
1
  • 1
    This is the same as the earlier suggestion by @nemisj No?
    – Hedgehog
    Feb 6, 2012 at 3:44
2
sed -i -e '1rmyheader' -e '1{h;d}' -e '2{x;G}' myfile
2

WARNING: this needs a bit more work to meet the OP's needs.

There should be a way to make the sed approach by @shixilun work despite his misgivings. There must be a bash command to escape whitespace when reading a file into a sed substitute string (e.g. replace newline characters with '\n'. Shell commands vis and cat can deal with nonprintable characters, but not whitespace, so this won't solve the OP's problem:

sed -i -e "1s/^/$(cat file_with_header.txt)/" file_to_be_prepended.txt

fails due to the raw newlines in the substitute script, which need to be prepended with a line continuation character () and perhaps followed by an &, to keep the shell and sed happy, like this SO answer

sed has a size limit of 40K for non-global search-replace commands (no trailing /g after the pattern) so would likely avoid the scary buffer overrun problems of awk that anonymous warned of.

2

You can use perl command line:

perl -i -0777 -pe 's/^/my_header/' tmp

Where -i will create an inline replacement of the file and -0777 will slurp the whole file and make ^ match only the beginning. -pe will print all the lines

Or if my_header is a file:

perl -i -0777 -pe 's/^/`cat my_header`/e' tmp

Where the /e will allow an eval of code in the substitution.

2

The simplest solution I found is:

cat myHeader myFile | tee myFile

or

echo "<line to add>" | cat - myFile | tee myFile

Notes:

  • Use echo -n if you want to append just a piece of text (not a full line).
  • Add &> /dev/null to the end if you don't want to see the output (the generated file).
  • This can be used to append a shebang to the file. Example:
    # make it executable (use u+x to allow only current user)
    chmod +x cropImage.ts
    # append the shebang
    echo '#''!'/usr/bin/env ts-node | cat - cropImage.ts | tee cropImage.ts &> /dev/null
    # execute it
    ./cropImage.ts myImage.png
    
1

If you're scripting in BASH, actually, you can just issue:

cat - yourfile  /tmp/out && mv /tmp/out yourfile

That's actually in the Complex Example you yourself posted in your own question.

1
  • An editor suggested changing this to cat - yourfile <<<"text" > /tmp/out && mv /tmp/out yourfile, however that's sufficiently different from my answer that it should be it's own answer. Jul 19, 2013 at 13:38
1

If you have a large file (few hundred kilobytes in my case) and access to python, this is much quicker than cat pipe solutions:

python -c 'f = "filename"; t = open(f).read(); open(f, "w").write("text to prepend " + t)'

1

A solution with printf:

new_line='the line you want to add'
target_file='/file you/want to/write to'

printf "%s\n$(cat ${target_file})" "${new_line}" > "${target_file}"

You could also do:

printf "${new_line}\n$(cat ${target_file})" > "${target_file}"

But in that case you have to be sure there aren’t any % anywhere, including the contents of target file, as that can be interpreted and screw up your results.

7
  • 2
    This seems to blow up (and truncate your file!) if you have anything in your target file that printf interprets as a formatting option.
    – Alan H.
    Aug 10, 2015 at 22:51
  • echo seems like a safer option. echo "my new line\n$(cat my/file.txt)" > my/file.txt
    – Alan H.
    Aug 10, 2015 at 22:58
  • @AlanH. I warn of the dangers in the answer.
    – user137369
    Aug 10, 2015 at 23:31
  • Actually, you only warned about percents in ${new_line}, not the target file
    – Alan H.
    Aug 10, 2015 at 23:53
  • Could you be more explicit? It’s a really big caveat IMO. “anywhere” doesn’t make this very clear.
    – Alan H.
    Aug 10, 2015 at 23:58
0
current=`cat my_file` && echo 'my_string' > my_file && echo $current >> my_file

where "my_file" is the file to prepend "my_string" to.

1
  • Unless you want some "magic" to start happening now and then, variable substitutions ought to be quoted. Sep 8, 2020 at 19:27
0

I'm liking @fluffle's ed approach the best. After all, any tool's command line switches vs scripted editor commands are essentially the same thing here; not seeing a scripted editor solution "cleanliness" being any lesser or whatnot.

Here's my one-liner appended to .git/hooks/prepare-commit-msg to prepend an in-repo .gitmessage file to commit messages:

echo -e "1r $PWD/.gitmessage\n.\nw" | ed -s "$1"

Example .gitmessage:

# Commit message formatting samples:
#       runlevels: boot +consolekit -zfs-fuse
#

I'm doing 1r instead of 0r, because that will leave the empty ready-to-write line on top of the file from the original template. Don't put an empty line on top of your .gitmessage then, you will end up with two empty lines. -s suppresses diagnostic info output of ed.

In connection with going through this, I discovered that for vim-buffs it is also good to have:

[core]
        editor = vim -c ':normal gg'
0

variables, ftw?

NEWFILE=$(echo deb http://mirror.csesoc.unsw.edu.au/ubuntu/ $(lsb_release -cs) main universe restricted multiverse && cat /etc/apt/sources.list)
echo "$NEWFILE" | sudo tee /etc/apt/sources.list
0

I think this is the cleanest variation of ed:

cat myheader | { echo '0a'; cat ; echo -e ".\nw";} | ed myfile

as a function:

function prepend() { { echo '0a'; cat ; echo -e ".\nw";} | ed $1; }

cat myheader | prepend myfile

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.