333

I have a bunch of log files. I need to find out how many times a string occurs in all files.

grep -c string *

returns

...
file1:1
file2:0
file3:0
...

Using a pipe I was able to get only files that have one or more occurrences:

grep -c string * | grep -v :0

...
file4:5
file5:1
file6:2
...

How can I get only the combined count? (If it returns file4:5, file5:1, file6:2, I want to get back 8.)

3
  • 1
    Can you tell me what the grep -v :0 does ? . I know it counts for files having occurrences greater than 0. What does the -v option and :0 mean ?. Kindly let me know. May 9, 2017 at 17:57
  • 2
    @GauthamHonnavara grep :0 looks for line that match the string :0. -v is an option to invert that search so instead using grep -v :0 means find all line that don't contain :0 so a line with file4:5 and file27:193 all would pass through since they don't contain :0
    – penguin359
    May 16, 2017 at 17:56
  • You can select multiple files using space. grep file1 file2 --options Sep 27, 2019 at 9:11

16 Answers 16

331

This works for multiple occurrences per line:

grep -o string * | wc -l
4
  • 2
    This also works: grep -o string * --exclude-dir=some/dir/one/ --exclude-dir=some/dir/two | wc -l.
    – a coder
    Nov 5, 2014 at 14:16
  • 3
    grep -ioR string * | wc -l is what I use to do a case-insensitive, recursive, matching-only search May 28, 2015 at 8:55
  • 3
    This one shows the relevant files and then the total count of matches: grep -rc test . | awk -F: '$NF > 0 {x+=$NF; $NF=""; print} END{print "Total:",x}'
    – Yaron
    Sep 6, 2017 at 11:40
  • Beware of limitations to grep: superuser.com/questions/1703029/…
    – duplex143
    Aug 24 at 16:50
303
cat * | grep -c string
7
  • 10
    This has the same limitation that it counts multiple occurrences on one line only once. I am guessing that this behavior is OK in this case, though. Dec 16, 2008 at 12:22
  • @Michael Haren Yes, there could be only one occurrence of string in a line. Dec 16, 2008 at 12:25
  • 2
    I'd rather do grep -c string<* So just replacing the space with a less than. Jan 4, 2012 at 2:08
  • 55
    Does not address multiple occurrences on a line
    – bluesman
    May 9, 2012 at 16:14
  • 2
    This doesn't work if you want to search in subdirectories too, whereas grep -o and wc -l does. cat is quicker in cases like the original question though. Jan 3, 2013 at 15:37
28
grep -oh string * | wc -w

will count multiple occurrences in a line

2
  • 30
    grep -oh "... my that curry was strong" * >> wc :)
    – icc97
    Mar 23, 2016 at 16:03
  • @icc97 did you mean to pipe to wc or to cwc? (curse word count)
    – Matiaan
    May 29 at 7:38
26

Instead of using -c, just pipe it to wc -l.

grep string * | wc -l

This will list each occurrence on a single line and then count the number of lines.

This will miss instances where the string occurs 2+ times on one line, though.

1
  • 2
    Piping to "wc -l" works also nicely together with "grep -r 'test' ." which scans recursively all files for the string 'test' in all directories below the current one. Dec 13, 2011 at 15:07
18
cat * | grep -c string

One of the rare useful applications of cat.

13

You can add -R to search recursively (and avoid to use cat) and -I to ignore binary files.

grep -RIc string .
13

If you want number of occurrences per file (example for string "tcp"):

grep -RIci "tcp" . | awk -v FS=":" -v OFS="\t" '$2>0 { print $2, $1 }' | sort -hr

Example output:

53  ./HTTPClient/src/HTTPClient.cpp
21  ./WiFi/src/WiFiSTA.cpp
19  ./WiFi/src/ETH.cpp
13  ./WiFi/src/WiFiAP.cpp
4   ./WiFi/src/WiFiClient.cpp
4   ./HTTPClient/src/HTTPClient.h
3   ./WiFi/src/WiFiGeneric.cpp
2   ./WiFi/examples/WiFiClientBasic/WiFiClientBasic.ino
2   ./WiFiClientSecure/src/ssl_client.cpp
1   ./WiFi/src/WiFiServer.cpp

Explanation:

  • grep -RIci NEEDLE . - looks for string NEEDLE recursively from current directory (following symlinks), ignoring binaries, counting number of occurrences, ignoring case
  • awk ... - this command ignores files with zero occurrences and formats lines
  • sort -hr - sorts lines in reverse order by numbers in first column

Of course, it works with other grep commands with option -c (count) as well. For example:

grep -c "tcp" *.txt | awk -v FS=":" -v OFS="\t" '$2>0 { print $2, $1 }' | sort -hr
1
  • Awesome! Worked like a charm. Save days time. Thank you so much. Jul 20, 2020 at 18:46
12

Obligatory AWK solution:

grep -c string * | awk 'BEGIN{FS=":"}{x+=$2}END{print x}'

Take care if your file names include ":" though.

12

Something different than all the previous answers:

perl -lne '$count++ for m/<pattern>/g;END{print $count}' *
1
  • nice to see an approach not using grep, esp as my grep (on windows) doesn't support the -o option. Mar 12, 2013 at 15:14
7

The AWK solution which also handles file names including colons:

grep -c string * | sed -r 's/^.*://' | awk 'BEGIN{}{x+=$1}END{print x}'

Keep in mind that this method still does not find multiple occurrences of string on the same line.

5

You can use a simple grep to capture the number of occurrences effectively. I will use the -i option to make sure STRING/StrING/string get captured properly.

Command line that gives the files' name:

grep -oci string * | grep -v :0

Command line that removes the file names and prints 0 if there is a file without occurrences:

grep -ochi string *
1
  • 1
    Could you please elaborate more your answer adding a little more description about the solution you provide?
    – abarisone
    Jun 12, 2015 at 13:27
5

short recursive variant:

find . -type f -exec cat {} + | grep -c 'string'
1
  • 1
    Thank you! Only your solution worked for me (summed the matches of all the files).
    – Nestor
    Aug 25, 2019 at 22:01
2

Grep only solution which I tested with grep for windows:

grep -ro "pattern to find in files" "Directory to recursively search" | grep -c "pattern to find in files"

This solution will count all occurrences even if there are multiple on one line. -r recursively searches the directory, -o will "show only the part of a line matching PATTERN" -- this is what splits up multiple occurences on a single line and makes grep print each match on a new line; then pipe those newline-separated-results back into grep with -c to count the number of occurrences using the same pattern.

2

Here is a faster-than-grep AWK alternative way of doing this, which handles multiple matches of <url> per line, within a collection of XML files in a directory:

awk '/<url>/{m=gsub("<url>","");total+=m}END{print total}' some_directory/*.xml

This works well in cases where some XML files don't have line breaks.

0

Another oneliner using basic command line functions handling multiple occurences per line.

 cat * |sed s/string/\\\nstring\ /g |grep string |wc -l
0
awk -v RS='' -v FPAT='fast' '{print NF,FILENAME}' <file1..N>

Take a string, make it a line look for instance of fast and then print the number of fields with the filename.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.