Linked Questions
16 questions linked to/from Python readlines() usage and efficient practice for reading
4
votes
1
answer
8k
views
Differences between file.read(), file.readline() and iterating over the file object [duplicate]
I am new to computer science and am trying to create a function in python that will open files on my computer.
I know that the function f.readline() grabs the current line as a string, but what ...
0
votes
2
answers
146
views
File to array in python [duplicate]
I'm reading a tutorial about python for very beginners and at some point the author define some ways to work with files. My doubt is related to memory management and file arrays.
#open a file for ...
16
votes
2
answers
26k
views
Using python ijson to read a large json file with multiple json objects
I'm trying to parse a large (~100MB) json file using ijson package which allows me to interact with the file in an efficient way. However, after writing some code like this,
with open(filename, 'r') ...
4
votes
4
answers
2k
views
Does for line in File read entire file
Does the following code read one line for each loop or does it read the entire file into memory first before beginning the iteration?
for line in f:
print(line)
My intentions are to read a ...
1
vote
4
answers
2k
views
How do I clean up empty lines in a configparser file?
I sometimes get an .ini file that looks like this after changing and deleting values:
[Section A]
x = 1
d = 2
[Section B]
a = 3
Is there an easy way to keep it clean and remove those blank lines ...
2
votes
3
answers
788
views
Does it takes RAM to save a readlines array?
I am using the command lineslist = file.readlines() of a 2GB file.
So, I guess it will create a lineslist array of 2GB or more size. So, basically is it the same as readfile = file.read(), which ...
2
votes
4
answers
1k
views
Does high disk usage mean faster file read/write operations?
I'm writing a python script in which I read a big file ~5 GB line by line, make some modifications in each of the lines, and then write it to another file.
When I use the function file.readlines() ...
1
vote
1
answer
2k
views
Python: iteratively add to dataframe
I have the following code:
for state in state_list:
state_df = pd.DataFrame()
for df in pd.read_csv(tax_sample,sep='\|\|', engine='python', dtype = tax_column_types, chunksize = 10, nrows = ...
0
votes
0
answers
1k
views
how to read a huge csv file using mmap in python?
I want to read csv file and perform some operation on that file. I'm created the program for my requirement but I'm not getting output because the file size is very large i.e. ~5GB.
I'm using simple ...
0
votes
3
answers
533
views
Unable to split the sequence from a Fasta File
my fasta file has multiple sequences like the following
>header1
MPANFTE
GSFDSSG
>header2
MDNASFS
EPWPANA
so I am writing a code to remove the headers and the output looks like this in a ...
1
vote
1
answer
350
views
What is causing this socket to run really slow?
the following code checks for an open port and the socket seems to be running really slow:
for line in f.readlines():
line = line.rstrip()
with closing(socket.socket(socket....
0
votes
3
answers
136
views
Locate multiple keywords in lines using Python
I got a line like this :
20:28:26.684597 24:d5:6e:76:9s:10 (oui Unknown) > 45:83:r4:7u:9s:i2
(oui Unknown), ethertype 802.1Q (0x8100), length 78: vlan 64, p 0,
ethertype IPv4, (tos 0x48, ttl 34,...
1
vote
3
answers
159
views
import dirty csv file with unwanted characters, strings
I would like to import csv files with pandas. Normally my data is given in the form:
a,b,c,d
a1,b1,c1,d1
a2,b2,c2,d2
where a,b,c,d is the header. I can easily use the pandas.read_csv here.
However, ...
0
votes
2
answers
136
views
Problem with python memory, flush, csv size
After solving a sorting of a dataset, I have a problem at this point of my code.
with open(fns_land[xx]) as infile:
lines = infile.readlines()
for line in lines:
result_station.append(...
-3
votes
1
answer
92
views
How to open a text file which has more than 500k lines, without using any iteration?
I am working with text files and looping over them, python works well with files of 10k to 20k lines, most of them are of that length, few text files are over 100k lines, where the code just stops or ...