I'm new to Linux/Unix. I find some task can be accomplished by many ways using these commands. Is there some relationship between them? Or which one is preferred? Which one is outdated?
Thanks.
I'm new to Linux/Unix. I find some task can be accomplished by many ways using these commands. Is there some relationship between them? Or which one is preferred? Which one is outdated?
Thanks.
See SO 366980 for discussion about the differences between Perl, Python, Awk and Sed.
There are four distinct commands in the list of five; awk and gawk are closely related, with GNU Awk being the GNU implementation of Awk.
find
is for locating files in a set of directories based on file characteristics such as name or modification time. GNU Find has many more capabilities than the traditional or POSIX versions of Find.
grep
is for locating content within files using regular expressions to control what is selected. GNU Grep has many more capabilities than the traditional or POSIX versions of Grep.
sed
is for modifying the contents of files using editing commands, including regular expressions. The GNU Sed has many more capabilities than the traditional or POSIX versions of Sed.
awk
is a pattern matching and formatting language. It is a programming language in a way that the other tools you mention are not. When needed, it is very useful. However, Perl and Python also have the capabilities of Awk and many extras, so many people use them instead of Awk. GNU Awk has some more capabilities than the traditional or POSIX versions of Awk.
So, the tools you list do different jobs, but can work together in many ways. One other tool you should be aware of is xargs
, which takes lists of files and runs a specified command on each of the files in turn.
expr
, test
, the sort
/uniq
/comm
/join
/cut
/paste
set, and so on…
Commented
Dec 21, 2010 at 8:32
AWK is a programming language designed by Aho, Weinberger, and Kernighan. gawk
is one implementation of AWK, but there are several others, including mawk
, and nawk
. It is a full-blown programming language with variables, control structures, and associative arrays, but generally optimized for dealing with the sorts of text-based data commonly found on UNIX systems.
sed
is the stream editor, inspired by ed
the editor. It has a simple command set mostly limited to line-by-line editing. sed
commands can easily by imitated in awk
. The following are equivalent:
sed -e 's/foo/bar/g'
awk '{ gsub(/foo/, "bar"); print $0 }'
awk '{gsub(/foo/,"bar")}1'
grep
finds text. Basic grep
functionality can easily be imitated in sed
and awk
. The following are equivalent:
grep 'foo.bar'
sed -n -e '/foo.bar/p'
awk '/foo.bar/ { print $0 }'
awk '/foo.bar/'
Oops, missed find
.
find
walks the filesystem tree, performing actions according to specified criterion. For example,
find . -name '.*' -prune -o ! -name '*~' -type f -exec cat '{}' \;
will walk all files and directories starting from the current directory .
, excluding (and not descending into) any directories with names starting with .
, and run cat
on every file whose name does not end with ~
(with the result of printing out the contents of that file). Again, this would be doable in AWK or Perl or many other programming languages (or even pure shell, in this example), but is easier and faster to write and understand with a special-purpose tool.
Different tools for different jobs. The question is like "what is a better tool: a hammer, a screwdriver or combination pliers?"
grep - find/filter text in a file/stream. Line matches/doesn't match - stream it to output. Use to reduce amount of output to what you need. Commonly used interactively if you get too much information. Also often used in scripts to pull "that one line containing what I need"
Usually after scripted use of grep you see either awk (either a complicated expression or just an old-school programmer) or cut (which is much faster but pretty simplistic) to extract the one value. (grep chops horizontally, cut/awk chops vertically)
Using awk for fully-featured programming is currently almost unseen.
sed - search-and-replace, usually scripted (a visual editor is much better for that if you want to do it interactively - you see what is being done.)
find - find files conforming to given (advanced) parameters.
These tools do different things, so which one is preferred depends on what you want to do.
Of the tools you mention, find
is a little different than the others -- it works on filesystems to search for files with certain attributes (name, date, permissions...).
sed
and grep
take text files as input, operate on them, and output the results. They both heavily use regular expressions. sed
("Stream EDitor") is generally used for search-and-replace operations. grep
("Global Regular Expression Print") outputs lines in a text file that match some pattern.
On Linux systems, awk
and gawk
are generally different names for the same program, the GNU version of AWK (named after its creators "Aho, Weinberger, Kernighan"). awk
can also refer to the "classic" version of awk
that doesn't have the GNU extensions, and there are other variants nawk
("new awk") and mawk
("Mike's AWK"). awk
does far more than pattern matching; it's really a full-blown programming language, though it's geared toward breaking up input text into lines and fields and operating on them (including numeric operations, so it's easy to do things like total up columns or do statistical calculations).
Listed in order of how often I use them, appended with comment. For more details, try man command
, eg man grep
at the command line for more details.
grep - print lines matching a pattern * I personally find this the most useful.
grep foo *.txt
grep 192.168.1.1 output.log
grep -i steve */*.txt
find - walk a file hierarchy * can be used to find files in subdirectories or execute shell commands on files
find . -name lost.txt -print
find . -name "*.txt" -print
find . -type f -exec chmod 0644 {} \;
sed - stream editor * helps automate simple text edits on batches of files
sed "s/abc/ABC/" foo.txt
awk - pattern-directed scanning and processing language * haven't used it for a long time. If it gets to the point I need to use awk, I generally use something like Ruby.