I want to traverse all subdirectories, except the node_modules
directory.
14 Answers
Recent versions of GNU Grep (>= 2.5.2) provide:
--exclude-dir=dir
which excludes directories matching the pattern dir
from recursive directory searches.
So you can do:
grep -R --exclude-dir=node_modules 'some pattern' /path/to/search
For a bit more information regarding syntax and usage see
- The GNU man page for File and Directory Selection
- A related StackOverflow answer Use grep --exclude/--include syntax to not grep through certain files
For older GNU Greps and POSIX Grep, use find
as suggested in other answers.
Or just use ack
(Edit: or The Silver Searcher) and be done with it!
-
5@Manocho: If you think
ack
is great, try The Silver Searcher and see the speed increase! Commented Nov 13, 2013 at 22:08 -
51Syntax for the impatient:
--exclude-dir=dir
usesgrep
's regular expression patterns, not shell's file globbing. Patterns work on paths relative to your current directory. So use pattern--exclude-dir=dir
, not--exclude-dir="/root/dir/*"
.– taniusCommented Feb 8, 2014 at 17:39 -
25If you wish to exclude multiple dirs from the search, is there a better option than to use :
$ grep -r --exclude-dir=dir1 --exclude-dir=dir2 "string" /path/to/search/dir
? Commented Nov 25, 2015 at 6:21 -
22I probably spent way too much time on this than any sane person, but I can't for the life of me figure out how to exclude a subdirectory from the search -
grep -r --exclude-dir=public keyword .
works, butgrep -r --exclude-dir='public/dist' keyword .
does not. I tried adding regex wildcards, escaping characters etc, but nothing seems to help.– dkobozevCommented Jul 6, 2016 at 23:47 -
117Exclude multiple directories like so:
grep -r "Request" . --exclude-dir={node_modules,git,build}
Commented Jul 12, 2016 at 8:00
SOLUTION 1 (combine find
and grep
)
The purpose of this solution is not to deal with grep
performance but to show a portable solution : should also work with busybox or GNU version older than 2.5.
Use find
, for excluding directories foo and bar :
find /dir \( -name foo -prune \) -o \( -name bar -prune \) -o -name "*.sh" -print
Then combine find
and the non-recursive use of grep
, as a portable solution :
find /dir \( -name node_modules -prune \) -o -name "*.sh" -exec grep --color -Hn "your text to find" {} 2>/dev/null \;
SOLUTION 2 (using the --exclude-dir
option of grep
):
You know this solution already, but I add it since it's the most recent and efficient solution. Note this is a less portable solution but more human-readable.
grep -R --exclude-dir=node_modules 'some pattern' /path/to/search
To exclude multiple directories, use --exclude-dir
as:
--exclude-dir={node_modules,dir1,dir2,dir3}
SOLUTION 3 (Ag)
If you frequently search through code, Ag (The Silver Searcher) is a much faster alternative to grep, that's customized for searching code. For instance, it automatically ignores files and directories listed in .gitignore
, so you don't have to keep passing the same cumbersome exclude options to grep
or find
.
-
3this combination searches faster than
--exclude-dir=dir
and it shows results with colors - easy to read Commented Oct 8, 2013 at 1:51 -
33"this combination"
find ... -exec
is not faster thangrep --exclude-dir
for me. Huge advantage to grep (about five times faster with 26k+ files, filtered out of 38k+ on an HDD), unless you replace the\;
with+
for the find/exec combo. Then grep is "only" about 30% faster. The grep syntax is also human-readble :). Commented Jan 27, 2014 at 17:48 -
Agreed, since this is obvious. Some busyboxes does not have the GREP command. Commented Feb 16, 2018 at 12:57
-
19also noting that you can exclude multiple with
--exclude-dir={dir1,dir2}
– suhCommented Nov 7, 2018 at 15:11 -
11I'm not the least bit surprised that
node_modules
is the canonical example. Commented Jan 16, 2019 at 18:21
If you want to exclude multiple directories:
"r" for recursive, "l" to print only names of files containing matches and "i" to ignore case distinctions :
grep -rli --exclude-dir={dir1,dir2,dir3} keyword /path/to/search
Example : I want to find files that contain the word 'hello'. I want to search in all my linux directories except proc directory, boot directory, sys directory and root directory :
grep -rli --exclude-dir={proc,boot,root,sys} hello /
Note : The example above needs to be root
Note 2 (according to @skplunkerin) : do not add spaces after the commas in {dir1,dir2,dir3}
-
10NOTE: do not add spaces after the commas in
{dir1,dir2,dir3}
Commented Jan 30, 2017 at 23:51 -
Thanks, handy when grep'ing through SVN workspace:
grep -Irsn --exclude-dir=.svn 'foo' .
– RAM237Commented Aug 8, 2017 at 15:41 -
2
This syntax
--exclude-dir={dir1,dir2}
is expanded by the shell (e.g. Bash), not by grep
, into this:
--exclude-dir=dir1 --exclude-dir=dir2
Quoting will prevent the shell from expanding it, so this won't work:
--exclude-dir='{dir1,dir2}' <-- this won't work
The patterns used with --exclude-dir
are the same kind of patterns described in the man page for the --exclude
option:
--exclude=GLOB
Skip files whose base name matches GLOB (using wildcard matching).
A file-name glob can use *, ?, and [...] as wildcards, and \ to
quote a wildcard or backslash character literally.
The shell will generally try to expand such a pattern itself, so to avoid this, you should quote it:
--exclude-dir='dir?'
You can use the curly braces and quoted exclude patterns together like this:
--exclude-dir={'dir?','dir??'}
If you are grepping for code in a git repository and node_modules
is in your .gitignore
, you can use git grep
. git grep
searches the tracked files in the working tree, ignoring everything from .gitignore
git grep "STUFF"
-
This is an incredible way to limit wading through tons of generated files (dist, bin, etc.) and it pipes into a utility like
less
automagically so you have a scrollable search result. Fantastic! Commented May 10, 2023 at 18:28
Many correct answers have been given here, but I'm adding this one to emphasize one point which caused some rushed attempts to fail before: exclude-dir
takes a pattern, not a path to a directory.
Say your search is:
grep -r myobject
And you notice that your output is cluttered with results from the src/other/objects-folder
. This command will not give you the intended result:
grep -r myobject --exclude-dir=src/other/objects-folder
And you may wonder why exclude-dir
isn't working! To actually exclude results from the objects-folder
, simply do this:
grep -r myobject --exclude-dir=objects-folder
In other words, just use the folder name, not the path. Obvious once you know it.
From the man page:
--exclude-dir=GLOB
Skip any command-line directory with a name suffix that matches the pattern GLOB. When searching recursively, skip any subdirectory whose base name matches GLOB. Ignore any redundant trailing slashes in GLOB.
-
3Why on the planet didn't I scroll down to this answer before I posted my comment/question up above? I unfortunately have the bad habit to ignore answers with less upvotes, but this explains what I was doing wrong, so thanks Nagev.– AndycCommented Apr 11, 2021 at 6:04
-
2I would go crazy because I used the full path to a subfolder, but it did not work. After I saw this comment, I used the folder name ONLY and it finally worked! THANKS– babis21Commented Apr 25, 2023 at 14:18
Frequently use this:
grep
can be used in conjunction with -r
(recursive), i
(ignore case) and -o
(prints only matching part of lines). To exclude files
use --exclude
and to exclude directories use --exclude-dir
.
Putting it together you end up with something like:
grep -rio --exclude={filenames comma separated} \
--exclude-dir={directory names comma separated} <search term> <location>
Describing it makes it sound far more complicated than it actually is. Easier to illustrate with a simple example.
Example:
Suppose I am searching for current project for all places where I explicitly set the string value debugger
during a debugging session, and now wish to review / remove.
I write a script called findDebugger.sh
and use grep
to find all occurrences. However:
For file exclusions - I wish to ensure that .eslintrc
is ignored (this actually has a linting rule about debugger
so should be excluded). Likewise, I don't want my own script to be referenced in any results.
For directory exclusions - I wish to exclude node_modules
as it contains lots of libraries that do reference debugger
and I am not interested in those results. Also I just wish to omit .idea
and .git
hidden directories because I don't care about those search locations either, and wish to keep the search performant.
So here is the result - I create a script called findDebugger.sh
with:
#!/usr/bin/env bash
grep -rio --exclude={.eslintrc,findDebugger.sh} \
--exclude-dir={node_modules,.idea,.git} debugger .
-
I believe the "r" option should be printed with an upper case "-R". Commented Mar 12, 2018 at 18:01
-
1Interesting. "r" has always worked for me on nix and mac. Commented Mar 13, 2018 at 2:34
-
When I wrote my answer, I used
-R
(I don't recall why now). I typically use-r
. It turns out that the uppercase version follows symlinks. TIL. Commented Apr 21, 2020 at 6:56 -
@Johnsyweb - thanks. upvoted your answer - don't recall when, likely in 2016 when I added this one :) Commented Apr 22, 2020 at 13:39
You could try something like grep -R search . | grep -v '^node_modules/.*'
-
45Not such a good solution in some cases. For example: If 'node_modules' directory is a huge one with lots of false positive matches (hence the need to filter out the directory) then the first grep is wasting a lot of time searching through a sub-directory and THEN the second grep filtering out the matches. It's faster to exclude node_modules in the first grep itself.– GuruMCommented Dec 13, 2012 at 8:22
-
2i don't care about the slowness, I can look at the command and know what it does– danschCommented Apr 22, 2014 at 18:39
-
2Ditto for Guru's comment. A grep of
/var
hangs when it hits/var/run
in my case. Hence the reason I want to avoid the directory in the first place.– jwwCommented Aug 31, 2015 at 9:48 -
6
Very useful, especially for those dealing with Node.js where we want to avoid searching inside "node_modules":
find ./ -not -path "*/node_modules/*" -name "*.js" | xargs grep keyword
Step 1:
vim ~/.bash_profile
search() {
grep -InH -r --exclude-dir=*build* -e "$1" .
}
Step 2:
source ~/.bash_profile
Usage:
search "<string_to_be_searched>"
A simple working command:
root/dspace# grep -r --exclude-dir={log,assetstore} "creativecommons.org"
Above I grep for text "creativecommons.org" in current directory "dspace" and exclude dirs {log,assetstore}.
Done.
This one works for me:
grep <stuff> -R --exclude-dir=<your_dir>
-
11How is this answer different from what has already been posted?– ᴍᴇʜᴏᴠCommented Aug 24, 2017 at 20:11
A simpler way would be to filter your results using "grep -v".
grep -i needle -R * | grep -v node_modules
-
16This is effectively the same answer DipSwitch provided 3 years earlier. It has the same problems, too.– jwwCommented Aug 31, 2015 at 9:53
node_modules
is in your.gitignore
,git grep "STUFF"
is the easiest way.git grep
searches the tracked files in the working tree, ignoring everything from.gitignore
grep -R --exclude-dir={node_modules,bower_components} "MyString" | cut -c1-"$COLUMNS"
-- further you could always alias this in the shell to 'nodegrep' or whatever and use a command argument as string input..