1150

I want to traverse all subdirectories, except the node_modules directory.

3
  • 3
    See superuser.com/q/66715/59933
    – borrible
    Commented Jul 3, 2011 at 20:55
  • 90
    If you are grepping for code in a git repository and node_modules is in your .gitignore, git grep "STUFF" is the easiest way. git grep searches the tracked files in the working tree, ignoring everything from .gitignore
    – 0xcaff
    Commented Dec 22, 2016 at 21:51
  • 10
    An example for node: grep -R --exclude-dir={node_modules,bower_components} "MyString" | cut -c1-"$COLUMNS" -- further you could always alias this in the shell to 'nodegrep' or whatever and use a command argument as string input..
    – B. Shea
    Commented Mar 21, 2018 at 21:59

14 Answers 14

1415

Recent versions of GNU Grep (>= 2.5.2) provide:

--exclude-dir=dir

which excludes directories matching the pattern dir from recursive directory searches.

So you can do:

grep -R --exclude-dir=node_modules 'some pattern' /path/to/search

For a bit more information regarding syntax and usage see

For older GNU Greps and POSIX Grep, use find as suggested in other answers.

Or just use ack (Edit: or The Silver Searcher) and be done with it!

20
  • 5
    @Manocho: If you think ack is great, try The Silver Searcher and see the speed increase!
    – johnsyweb
    Commented Nov 13, 2013 at 22:08
  • 51
    Syntax for the impatient: --exclude-dir=dir uses grep's regular expression patterns, not shell's file globbing. Patterns work on paths relative to your current directory. So use pattern --exclude-dir=dir, not --exclude-dir="/root/dir/*".
    – tanius
    Commented Feb 8, 2014 at 17:39
  • 25
    If you wish to exclude multiple dirs from the search, is there a better option than to use : $ grep -r --exclude-dir=dir1 --exclude-dir=dir2 "string" /path/to/search/dir ? Commented Nov 25, 2015 at 6:21
  • 22
    I probably spent way too much time on this than any sane person, but I can't for the life of me figure out how to exclude a subdirectory from the search - grep -r --exclude-dir=public keyword . works, but grep -r --exclude-dir='public/dist' keyword . does not. I tried adding regex wildcards, escaping characters etc, but nothing seems to help.
    – dkobozev
    Commented Jul 6, 2016 at 23:47
  • 117
    Exclude multiple directories like so: grep -r "Request" . --exclude-dir={node_modules,git,build}
    – maverick97
    Commented Jul 12, 2016 at 8:00
621

SOLUTION 1 (combine find and grep)

The purpose of this solution is not to deal with grep performance but to show a portable solution : should also work with busybox or GNU version older than 2.5.

Use find, for excluding directories foo and bar :

find /dir \( -name foo -prune \) -o \( -name bar -prune \) -o -name "*.sh" -print

Then combine find and the non-recursive use of grep, as a portable solution :

find /dir \( -name node_modules -prune \) -o -name "*.sh" -exec grep --color -Hn "your text to find" {} 2>/dev/null \;

SOLUTION 2 (using the --exclude-dir option of grep):

You know this solution already, but I add it since it's the most recent and efficient solution. Note this is a less portable solution but more human-readable.

grep -R --exclude-dir=node_modules 'some pattern' /path/to/search

To exclude multiple directories, use --exclude-dir as:

--exclude-dir={node_modules,dir1,dir2,dir3}

SOLUTION 3 (Ag)

If you frequently search through code, Ag (The Silver Searcher) is a much faster alternative to grep, that's customized for searching code. For instance, it automatically ignores files and directories listed in .gitignore, so you don't have to keep passing the same cumbersome exclude options to grep or find.

8
  • 3
    this combination searches faster than --exclude-dir=dir and it shows results with colors - easy to read Commented Oct 8, 2013 at 1:51
  • 33
    "this combination" find ... -exec is not faster than grep --exclude-dir for me. Huge advantage to grep (about five times faster with 26k+ files, filtered out of 38k+ on an HDD), unless you replace the \; with + for the find/exec combo. Then grep is "only" about 30% faster. The grep syntax is also human-readble :). Commented Jan 27, 2014 at 17:48
  • Agreed, since this is obvious. Some busyboxes does not have the GREP command.
    – hornetbzz
    Commented Feb 16, 2018 at 12:57
  • 19
    also noting that you can exclude multiple with --exclude-dir={dir1,dir2}
    – suh
    Commented Nov 7, 2018 at 15:11
  • 11
    I'm not the least bit surprised that node_modules is the canonical example. Commented Jan 16, 2019 at 18:21
127

If you want to exclude multiple directories:

"r" for recursive, "l" to print only names of files containing matches and "i" to ignore case distinctions :

grep -rli --exclude-dir={dir1,dir2,dir3} keyword /path/to/search

Example : I want to find files that contain the word 'hello'. I want to search in all my linux directories except proc directory, boot directory, sys directory and root directory :

grep -rli --exclude-dir={proc,boot,root,sys} hello /

Note : The example above needs to be root

Note 2 (according to @skplunkerin) : do not add spaces after the commas in {dir1,dir2,dir3}

3
  • 10
    NOTE: do not add spaces after the commas in {dir1,dir2,dir3} Commented Jan 30, 2017 at 23:51
  • Thanks, handy when grep'ing through SVN workspace: grep -Irsn --exclude-dir=.svn 'foo' .
    – RAM237
    Commented Aug 8, 2017 at 15:41
  • 2
    You can just provide the --exclude-dir option multiple times.
    – Walf
    Commented Aug 15, 2017 at 1:52
126

This syntax

--exclude-dir={dir1,dir2}

is expanded by the shell (e.g. Bash), not by grep, into this:

--exclude-dir=dir1 --exclude-dir=dir2

Quoting will prevent the shell from expanding it, so this won't work:

--exclude-dir='{dir1,dir2}'    <-- this won't work

The patterns used with --exclude-dir are the same kind of patterns described in the man page for the --exclude option:

--exclude=GLOB
    Skip files whose base name matches GLOB (using wildcard matching).
    A file-name glob can use *, ?, and [...]  as wildcards, and \ to
    quote a wildcard or backslash character literally.

The shell will generally try to expand such a pattern itself, so to avoid this, you should quote it:

--exclude-dir='dir?'

You can use the curly braces and quoted exclude patterns together like this:

--exclude-dir={'dir?','dir??'}
32

If you are grepping for code in a git repository and node_modules is in your .gitignore, you can use git grep. git grep searches the tracked files in the working tree, ignoring everything from .gitignore

git grep "STUFF"
1
  • This is an incredible way to limit wading through tons of generated files (dist, bin, etc.) and it pipes into a utility like less automagically so you have a scrollable search result. Fantastic!
    – Blake Neal
    Commented May 10, 2023 at 18:28
25

Many correct answers have been given here, but I'm adding this one to emphasize one point which caused some rushed attempts to fail before: exclude-dir takes a pattern, not a path to a directory.

Say your search is:

grep -r myobject

And you notice that your output is cluttered with results from the src/other/objects-folder. This command will not give you the intended result:

grep -r myobject --exclude-dir=src/other/objects-folder

And you may wonder why exclude-dir isn't working! To actually exclude results from the objects-folder, simply do this:

grep -r myobject --exclude-dir=objects-folder

In other words, just use the folder name, not the path. Obvious once you know it.

From the man page:

--exclude-dir=GLOB
Skip any command-line directory with a name suffix that matches the pattern GLOB. When searching recursively, skip any subdirectory whose base name matches GLOB. Ignore any redundant trailing slashes in GLOB.

2
  • 3
    Why on the planet didn't I scroll down to this answer before I posted my comment/question up above? I unfortunately have the bad habit to ignore answers with less upvotes, but this explains what I was doing wrong, so thanks Nagev.
    – Andyc
    Commented Apr 11, 2021 at 6:04
  • 2
    I would go crazy because I used the full path to a subfolder, but it did not work. After I saw this comment, I used the folder name ONLY and it finally worked! THANKS
    – babis21
    Commented Apr 25, 2023 at 14:18
17

Frequently use this:

grep can be used in conjunction with -r (recursive), i (ignore case) and -o (prints only matching part of lines). To exclude files use --exclude and to exclude directories use --exclude-dir.

Putting it together you end up with something like:

grep -rio --exclude={filenames comma separated} \
--exclude-dir={directory names comma separated} <search term> <location>

Describing it makes it sound far more complicated than it actually is. Easier to illustrate with a simple example.

Example:

Suppose I am searching for current project for all places where I explicitly set the string value debugger during a debugging session, and now wish to review / remove.

I write a script called findDebugger.sh and use grep to find all occurrences. However:

For file exclusions - I wish to ensure that .eslintrc is ignored (this actually has a linting rule about debugger so should be excluded). Likewise, I don't want my own script to be referenced in any results.

For directory exclusions - I wish to exclude node_modules as it contains lots of libraries that do reference debugger and I am not interested in those results. Also I just wish to omit .idea and .git hidden directories because I don't care about those search locations either, and wish to keep the search performant.

So here is the result - I create a script called findDebugger.sh with:

#!/usr/bin/env bash
grep -rio --exclude={.eslintrc,findDebugger.sh} \
--exclude-dir={node_modules,.idea,.git} debugger .
4
  • I believe the "r" option should be printed with an upper case "-R".
    – hornetbzz
    Commented Mar 12, 2018 at 18:01
  • 1
    Interesting. "r" has always worked for me on nix and mac.
    – arcseldon
    Commented Mar 13, 2018 at 2:34
  • When I wrote my answer, I used -R (I don't recall why now). I typically use -r. It turns out that the uppercase version follows symlinks. TIL.
    – johnsyweb
    Commented Apr 21, 2020 at 6:56
  • @Johnsyweb - thanks. upvoted your answer - don't recall when, likely in 2016 when I added this one :)
    – arcseldon
    Commented Apr 22, 2020 at 13:39
6

You could try something like grep -R search . | grep -v '^node_modules/.*'

4
  • 45
    Not such a good solution in some cases. For example: If 'node_modules' directory is a huge one with lots of false positive matches (hence the need to filter out the directory) then the first grep is wasting a lot of time searching through a sub-directory and THEN the second grep filtering out the matches. It's faster to exclude node_modules in the first grep itself.
    – GuruM
    Commented Dec 13, 2012 at 8:22
  • 2
    i don't care about the slowness, I can look at the command and know what it does
    – dansch
    Commented Apr 22, 2014 at 18:39
  • 2
    Ditto for Guru's comment. A grep of /var hangs when it hits /var/run in my case. Hence the reason I want to avoid the directory in the first place.
    – jww
    Commented Aug 31, 2015 at 9:48
  • 6
    --exclude-dir is the best solution as of 2016.
    – Omar Tariq
    Commented Dec 13, 2016 at 22:00
4

Very useful, especially for those dealing with Node.js where we want to avoid searching inside "node_modules":

find ./ -not -path "*/node_modules/*" -name "*.js" | xargs grep keyword
3

Step 1:

vim ~/.bash_profile

search() {
    grep -InH -r --exclude-dir=*build* -e "$1" .
}

Step 2:

source ~/.bash_profile

Usage:

search "<string_to_be_searched>"

2

A simple working command:

root/dspace# grep -r --exclude-dir={log,assetstore} "creativecommons.org"

Above I grep for text "creativecommons.org" in current directory "dspace" and exclude dirs {log,assetstore}.

Done.

2
  • Neat, including several directories in brackets
    – Mijo
    Commented Nov 15, 2017 at 19:49
  • This was already mentioned in three existing answers. Suggest deletion to clean up this thread and remove noise/redundancy.
    – ggorlen
    Commented Mar 12 at 17:10
0
find . ! -name "node_modules" -type d 
0
-1

This one works for me:

grep <stuff> -R --exclude-dir=<your_dir>
1
  • 11
    How is this answer different from what has already been posted? Commented Aug 24, 2017 at 20:11
-6

A simpler way would be to filter your results using "grep -v".

grep -i needle -R * | grep -v node_modules

1
  • 16
    This is effectively the same answer DipSwitch provided 3 years earlier. It has the same problems, too.
    – jww
    Commented Aug 31, 2015 at 9:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.