10

I'm new to Linux/Unix. I find some task can be accomplished by many ways using these commands. Is there some relationship between them? Or which one is preferred? Which one is outdated?

Thanks.

5
  • 1
    Isn't there a Unix user stackexchange site where you can ask questions like this?
    – Gabe
    Commented Dec 21, 2010 at 8:04
  • 3
    @Gabe: it is a perfectly fine question for SO - this is about programming, and the question is about which commands are useful for which parts of shell programming. Commented Dec 21, 2010 at 8:12
  • Jonathan: Sure, you can consider almost anything in computers to be a programming task. I just thought that since this is a "describe the tools" type of question rather than a "how do I solve this problem?" type of question, it might get better answers on a dedicated Unix site.
    – Gabe
    Commented Dec 21, 2010 at 8:14
  • 1
    @Gabe: it'll get plenty of good answers here - there are Unix programmers around who'll provide them. This is NOT a Windows-only site! Commented Dec 21, 2010 at 8:24
  • Jonathan: I just think that questions like "what's the relationship between them?" and "which one is preferred?" are too general or subjective for the standards of this site. I would say the same thing if the question was "C, C++, C#" or "vi, emacs, IDE" and those are much more programmer-specific.
    – Gabe
    Commented Dec 21, 2010 at 16:01

5 Answers 5

12

See SO 366980 for discussion about the differences between Perl, Python, Awk and Sed.

There are four distinct commands in the list of five; awk and gawk are closely related, with GNU Awk being the GNU implementation of Awk.

  • find is for locating files in a set of directories based on file characteristics such as name or modification time. GNU Find has many more capabilities than the traditional or POSIX versions of Find.

  • grep is for locating content within files using regular expressions to control what is selected. GNU Grep has many more capabilities than the traditional or POSIX versions of Grep.

  • sed is for modifying the contents of files using editing commands, including regular expressions. The GNU Sed has many more capabilities than the traditional or POSIX versions of Sed.

  • awk is a pattern matching and formatting language. It is a programming language in a way that the other tools you mention are not. When needed, it is very useful. However, Perl and Python also have the capabilities of Awk and many extras, so many people use them instead of Awk. GNU Awk has some more capabilities than the traditional or POSIX versions of Awk.

So, the tools you list do different jobs, but can work together in many ways. One other tool you should be aware of is xargs, which takes lists of files and runs a specified command on each of the files in turn.

1
  • 1
    Good one. Other tools useful for shell use might include expr, test, the sort/uniq/comm/join/cut/paste set, and so on…
    – ephemient
    Commented Dec 21, 2010 at 8:32
5

AWK is a programming language designed by Aho, Weinberger, and Kernighan. gawk is one implementation of AWK, but there are several others, including mawk, and nawk. It is a full-blown programming language with variables, control structures, and associative arrays, but generally optimized for dealing with the sorts of text-based data commonly found on UNIX systems.

sed is the stream editor, inspired by ed the editor. It has a simple command set mostly limited to line-by-line editing. sed commands can easily by imitated in awk. The following are equivalent:

sed -e 's/foo/bar/g'
awk '{ gsub(/foo/, "bar"); print $0 }'
awk '{gsub(/foo/,"bar")}1'

grep finds text. Basic grep functionality can easily be imitated in sed and awk. The following are equivalent:

grep 'foo.bar'
sed -n -e '/foo.bar/p'
awk '/foo.bar/ { print $0 }'
awk '/foo.bar/'

Oops, missed find.

find walks the filesystem tree, performing actions according to specified criterion. For example,

find . -name '.*' -prune -o ! -name '*~' -type f -exec cat '{}' \;

will walk all files and directories starting from the current directory ., excluding (and not descending into) any directories with names starting with ., and run cat on every file whose name does not end with ~ (with the result of printing out the contents of that file). Again, this would be doable in AWK or Perl or many other programming languages (or even pure shell, in this example), but is easier and faster to write and understand with a special-purpose tool.

4

Different tools for different jobs. The question is like "what is a better tool: a hammer, a screwdriver or combination pliers?"

grep - find/filter text in a file/stream. Line matches/doesn't match - stream it to output. Use to reduce amount of output to what you need. Commonly used interactively if you get too much information. Also often used in scripts to pull "that one line containing what I need"

Usually after scripted use of grep you see either awk (either a complicated expression or just an old-school programmer) or cut (which is much faster but pretty simplistic) to extract the one value. (grep chops horizontally, cut/awk chops vertically)

Using awk for fully-featured programming is currently almost unseen.

sed - search-and-replace, usually scripted (a visual editor is much better for that if you want to do it interactively - you see what is being done.)

find - find files conforming to given (advanced) parameters.

3

These tools do different things, so which one is preferred depends on what you want to do.

Of the tools you mention, find is a little different than the others -- it works on filesystems to search for files with certain attributes (name, date, permissions...).

sed and grep take text files as input, operate on them, and output the results. They both heavily use regular expressions. sed ("Stream EDitor") is generally used for search-and-replace operations. grep ("Global Regular Expression Print") outputs lines in a text file that match some pattern.

On Linux systems, awk and gawk are generally different names for the same program, the GNU version of AWK (named after its creators "Aho, Weinberger, Kernighan"). awk can also refer to the "classic" version of awk that doesn't have the GNU extensions, and there are other variants nawk ("new awk") and mawk ("Mike's AWK"). awk does far more than pattern matching; it's really a full-blown programming language, though it's geared toward breaking up input text into lines and fields and operating on them (including numeric operations, so it's easy to do things like total up columns or do statistical calculations).

2

Listed in order of how often I use them, appended with comment. For more details, try man command, eg man grep at the command line for more details.

grep - print lines matching a pattern * I personally find this the most useful.

grep foo *.txt
grep 192.168.1.1 output.log
grep -i steve */*.txt 

find - walk a file hierarchy * can be used to find files in subdirectories or execute shell commands on files

find . -name lost.txt -print
find . -name "*.txt" -print
find . -type f -exec chmod 0644 {} \;

sed - stream editor * helps automate simple text edits on batches of files

sed "s/abc/ABC/" foo.txt

awk - pattern-directed scanning and processing language * haven't used it for a long time. If it gets to the point I need to use awk, I generally use something like Ruby.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.