2

I am trying to search text from one file (named as temporal3) in another file (names as map) using shell's grep function. Here is an small part of these files:

temporal3:

TACTATATTACTGGAAAAACCATCAAGCTAGACCA
AATATGTCCCAGTTGAAGAGAATTATCATTTTGAG
AGAATTATCATTTTGAGATTGCTATCGTTCACCAA
AATATGTCCCAGTTGAAGAGAATTATCATTTTGAG
CTAGACCAATATAATATGTCCCAGTTGAAGAGAAT
AGATTGCTATCGTTCACCAAATACTGGCAAATAAT

map:

CTTCTGAATCCGCCAGGAGTTCTACCTCGATCAAATAATGACACTTTGCACTTGATTTTTCATCAAGAAAGTCAGTCAAATAGTTCCAGAAATGGACACAA
ATCTCACATCCCTTACTATATTACTGGAAAAACCATCAAGCTAGACCAATATAATATATCCCAGTTGAAGAGAATTATCATTTTGAGATTGCTATCGTTCA
TCACATCCCTTACTATATTACTGGAAAAACCATCAAGCTAGACCAATATAATATGTCCCAGTTGAAGAGAATTATCATTTTGAGATTGCTATCGTTCACCA
AGACCAATATAATATGTCCCAGTTGAAGAGAATTATCATTTTGAGATTGCTATCGTTCACCAAATACTGGCAAATAATCTCCTTCTGAATCCGCCAGGAGT
CCAGTTGAAGAGAATTATCATTTTGAGATTGCTATCGTTCACCAAATACTGGCAAATAATCTCCTTCTGAATCCGCCAGGAGTTCTACCTCGATCAAATAA
GAGATTCATCTCACATCCCTTACTATATTACTGGAAAAACCATCAAGCTAGACCAATATAATATGTCCCAGTTGAAGAGAATTATCATTTTGAGATTGCTA
AGACCAATATAATATGTCCCAGTTGAAGAGAATTATCATTTTGAGATTGCTATCGTTCACCAAATACTGGCAAATAATCTCCTTCTGAATCCGCCAGGAGT
AACCATCAAGCTAGACCAATATAATATGTCCCAGTTGAAGAGAATTATCATTTTGAGATTGCTATCGTTCACCAAATACTGGCAAATAATCTCCTTCTGAA
TTCATCTCACATCCCTTACTATATTACTGGAAAAACCATCAAGCTAGACCAATATAATATGTCCCAGTTGAAGAGAATTATCATTTTGAGATTGCTATCGT
ATAGCGAGATTCATCTCGCATCCCTTACTATATTACTGGAAAAACCATCAAGCTAGACCAATATAATATGTCCCAGTTGAAGAGAATTATCATTTTGAGAT

The real map file is a hugh one (~20Gb) so I cannot handle it with R. What I want to do is:

  1. For the first line in temporal3, I want to get all lines in map that contain it.
  2. For the second line in temporal3, I want to get all lines in map that contain it.
  3. ...
  4. For the last line in temporal3, I want to get all lines in map that contain it.

I tried grep -Ff temporal3 map but it did not work! This is really surprising for me. Any idea about why this does not work? Any suggestions to get this task done? Many thanks!

5
  • Can you check your file for carriage returns? See the bash tag wiki step 1. May 14, 2014 at 16:30
  • Aha, indeed, they are in DOS style. After change them into unix style. The grep works. But returned one file for all lines in temproal3 file. How can I get get one output for each line in the temporal3? Maybe need for loop? Thanks! May 14, 2014 at 16:41
  • Yes you will need while read loop.
    – anubhava
    May 14, 2014 at 16:43
  • Can you give a workable code example? I am new to bash... Thanks! May 14, 2014 at 16:46
  • grep -Ff <(sed 's/\r$//' temporal3) map will remove the trailing newlines. May 14, 2014 at 17:05

2 Answers 2

1

You can use:

#!/bin/bash

> output
while read -r line; do
    grep -F "$line" map >> output
done < temporal3
2
  • What is the -r option doing here? May 14, 2014 at 17:34
  • @DaijiangLi: -r do not allow backslashes to escape any characters
    – anubhava
    May 14, 2014 at 17:35
1

Thanks @anubhava ! This is what I want. I also changed it a little bit:

 #!/bin/bash

 > output
 while read -r line; do
     echo $line >> output
     grep -Fn "$line" map >> output
 done < temporal3

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.