4

I am using the python package mailbox, and I am trying to extract the messages and clean the data. I am running into the problem that for large databases, I can call the constructor with my sample file, but when I try to print any messages my program hangs. I assume it is because the file I am trying to read is over 7GB. How can I deal with this problem?

1 Answer 1

0

Consider splitting the mailbox manually. The format is fairly easy to process (as long as you only need read-only access) by reading it line-per-line; and you can use the existing classes for the actual parsing of individual messages.

Look up the definition of the mbox format - lines beginning with "From" start a new mail. You can split the huge file at these markers, then use the mailbox package to read only one file at a time.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.