0

I have the following working script to grep in a directory of Many files from some specific strings previously saved into a file.

I use the files extension to grep all files as its name are random and note that every string from my previously file should be searched in all the files.

Also, I cut the outputting grep as it return 2 or 3 lines of the matched file and I only want a specific part that shows the filename.

I might be using something redundant, how it could be faster?

#!/bin/bash
#working but slow
cd /var/FILES_DIRECTORY
while read line
do
LC_ALL=C fgrep "$line" *.cps | cut -c1-27 >> /var/tmp/test_OUT.txt
done < "/var/tmp/test_STRINGS.txt"
3
  • There are numerous questions here on S.O. about making grep run faster with multiple strings, so take the time to read those msgs. Also, you may be able to make your question more understandable to a wider range of readers by eliminating the verbal description (I want this, and this and this, and I'm doing this and this and this), by using 1 simplified sample data, 2. the required output given that input data, and 3. possibly include a 2nd round of 1. and 2. to illustrate your 2nd requirement "note that every string from my previously file should be searched in all the files". Good luck.
    – shellter
    Sep 26, 2013 at 18:16
  • Thanks for your comments, I keep this in mind. Sep 27, 2013 at 15:18
  • I am using solaris 11 Oct 2, 2013 at 20:17

2 Answers 2

4
grep -F -f /var/tmp/test_STRINGS.txt *.cps | cut -c1-27

Isn't what you're looking for ?

2
  • It didn´t worked for me, grep: illegal option -- F grep: illegal option -- f Usage: grep -hblcnsviw pattern file Oct 1, 2013 at 18:09
  • What distro / version are you on ? Both -F and -f are part of POSIX. Oct 2, 2013 at 16:10
0

this should speed up your script :

#!/bin/bash
#working  fast 

cd /var/FILES_DIRECTORY
export LC_ALL=C

 grep -f /var/tmp/test_STRINGS.txt *.cps | cut -c1-27 > /var/tmp/test_OUT.txt
5
  • click the question mark icon above the top-right corner of the editing textbox to get markdown help. Sep 26, 2013 at 19:22
  • But I could use the same structure reading line by line or its not necessary? Oct 1, 2013 at 18:14
  • notice there is only on open/close output file in /var/tmp/test_OUT.txt <b> also grep will use a table (-f file ) and is executed only once. for large number of files I think my answer is faster.
    – michael501
    Oct 1, 2013 at 18:19
  • I am facing the same problem as the first answer, when using grep -f. I implement your function inside the read line structure and get back to you. Oct 1, 2013 at 18:21
  • Inside read line by line structure it still slow. I could improve by looking for each day files as this FILES DIRECTORY is very big. But looking for yesterday I have a particular big number of files anyway. Oct 1, 2013 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.