I'm trying to substitute only the first match of one string in a huge file with only one line (2.1 GB), this substitution will occur in a shell script job. The big problem is that the machine that will run this script has only 1GB memory (approximately
300MB free), so i need a buffered strategy that don't overflow my memory. I already tried sed
, perl
and a python
approach, but all of them return me out of memory errors.
Here are my attemps (discovered in other questions):
# With perl
perl -pi -e '!$x && s/FROM_STRING/TO_STRING/ && ($x=1)' file.txt
# With sed
sed '0,/FROM_STRING/s//TO_STRING/' file.txt > file.txt.bak
# With python (in a custom script.py file)
for line in fileinput.input('file.txt', inplace=True):
print line.replace(FROM_STRING, TO_STRING, 1)
break
One good point is that the FROM_STRING
that i'm searching is always in the beggining of this huge 1-line file, in the first 100 characters. Other good thing is that the execution time is not a problem, it can take time without problems.
EDIT (SOLUTION):
I tested three solutions of the answers, all them solved the problem, thanks for all of you. I tested the performance with Linux time
and all of them take about the same time too, up to 10 seconds approximately... But i choose the @Miller solution because it's simpler (just uses perl
).