Questions tagged [text-processing]

Mechanizing the creation or manipulation of electronic text.

Filter by
Sorted by
Tagged with
-3 votes
0 answers
27 views

how to extract line items from unstructured invoice pdf and make datafarme [closed]

I want to extract line items from invoice pdf, items name and description may be multiline. here is my extracted text text = """ Resistor 0 25 1 OHM RESISTOR Edgewound 11.48 287.00 EWR2-...
shivani's user avatar
  • 79
-3 votes
1 answer
90 views

Extract N lines with no duplicate strings from either of the two first columns

I have a huge CSV file with pairs of unique combinations. I need to extract N random lines of this file, but I need that the strings in col 1&2 to be unique, so that the list of unique strings in ...
MrLungo's user avatar
  • 37
-1 votes
0 answers
10 views

How to process texts on PC screen in real time?

I want to process the texts on PC screen in real time by following the mouse pointer when I have multiple monitor setup or the primary screen when I have only one monitor. So, I had the idea that I ...
rng70's user avatar
  • 23
1 vote
1 answer
40 views

Using a Word Counter in Python is understating results

As a complete preface, I am a beginner and learning. But, here's the sample schema of my products review table. Record_ID Product_ID Review Comment 1234 89847457 I love this product it was shipped ...
user14452102's user avatar
0 votes
0 answers
42 views

How to Add Product ID Search Functionality to Python Word Counter and Display Associated Words?

I have a very large list of Movie Reviews. This is my schema sample...it's about 4000 rows, so there's a lot going on. MovieID FreeformComment 1234 This was a great movie it made me happy 5678 This ...
user14452102's user avatar
-2 votes
1 answer
41 views

How to merge some elements in a python list based on some features

This is a list, each elements consists of two strings and a "/t" between.We can call the string on the left "label" and the part on the right "text". continued In the ...
Haochuan Liu's user avatar
-2 votes
1 answer
50 views

Sed copy first string and prepend it to line [duplicate]

I have a file: Text1 text2 50 Text3 text4 60 I would like to make the following using sed command: Text1 Text1 text2 50 Text3 Text3 text4 60 I need to copy the first string and add it to the line.
user367640's user avatar
-2 votes
2 answers
106 views

How can I find out the number of tabs at the beginning of each line in a text file?

I have a text file, where each line may start with a number of tabs, including no tabs. For example, the first line starts with no tab, the second line with 1 tab, and the third line with 2 tabs: ...
Tim's user avatar
  • 95.6k
1 vote
2 answers
103 views

Athena/Trino/Presto code to parse text file with custom line separator

I want to parse some log files in plain text file with records spanning across multiple lines using Trino/Presto code. My data looks like this: each record has multiple lines and each line has a ...
Jason's user avatar
  • 1,122
1 vote
1 answer
100 views

SAM alignment: Extract specific region in the query sequence and its enclosing parts in the CIGAR string

I need to perform the local alignment of a given region of a DNA sequence for which a global alignment was made, and update the corresponding part of the global CIGAR string. The steps would be as ...
Fravadona's user avatar
  • 14.4k
0 votes
1 answer
57 views

How to reliably intercept (and cancel) all the input in contenteditable?

I need to catch all the input in my contenteditable element and process the input in the code. Basically, what I need is to prevent input event from changing data in element and to know what data was ...
Andrew's user avatar
  • 835
1 vote
1 answer
109 views

Text processing with Python

I need to extract 1,500,000 out of 25,000,000 records and group them. The groups and the UUIDs of the records to extract are defined in a separate file (200MB) with the following format: >Cluster 0 ...
Fravadona's user avatar
  • 14.4k
-1 votes
1 answer
28 views

Processing through files in Python

Good morning fellow coders. I have a question regarding the use of finding specific lines in a file using Python. One method is to use if line.startswith(word) or to use if not line.startswith(word) ...
TheMethodicalWriter's user avatar
1 vote
1 answer
71 views

How to split an array of string nodes at given index?

I probably need to use other data structure, but I'm stuck with this solution for now. Will appreciate any advice on this. For now I have this data structure: const data = [ { id: 'node-1',...
Andrew's user avatar
  • 835
0 votes
1 answer
29 views

How to 'normalize' an array of objects?

I am not sure if the term 'normalization' is correct in my case, so I would appreciate if you point me at the right one. I found this term in DOM node method called normalize, which is doing kinda the ...
Andrew's user avatar
  • 835
2 votes
3 answers
224 views

AWS Textract OCR reading PDF as single line instead of preserving line breaks

Hi I am new to AWS Textract. I am using Amazon Textract to extract text from a PDF file. However, the output is not preserving the line breaks from the original PDF. For example, in the PDF there are ...
tt0206's user avatar
  • 779
0 votes
1 answer
47 views

Conversion of data to perform logistic regression

I have a dataset which contains the data of cricket matches from 2010 to 2023. I have attached a sample of the data for reference: Team_1 Team_2 Ground Matchdate Bangladesh Srilanka Mirpur 2010-01-...
Yash Salvi's user avatar
3 votes
3 answers
44 views

Counting string occurrences in second column which corresponds to first columns of a file [closed]

I have this Input text file: CD196_RS15035 normal alleles CD196_RS15035 normal alleles CD196_RS15035 truncation in the allele CD196_RS15035 truncation in the allele ...
user14714429's user avatar
1 vote
0 answers
74 views

How to handle ligature issue while using pdf text

I need to capture some text from some PDFs. I use PymuPDF to do this. But facing ligature issue while writing those selected text inside a text file. I use the following code snippet to read the PDF ...
WhyMeasureTheory's user avatar
-2 votes
1 answer
55 views

How to compress multiple lines into single/fewer lines using regular expression or wildcard character?

I want to compress multiple lines of string having some common elements into single or fewer lines by replacing the changing values with wildcard character. Example Input: Lorem ipsum dolor sit amet_0 ...
Saurabh's user avatar
1 vote
1 answer
91 views

Can't find model 'en_core_web_lg'. It doesn't seem to be a Python package or a valid path to a data directory. Eventhough they are in same directory

I am trying different text processing model. I am trying to use spacy and it's model en_core_web_lg. import spacy import spacy.language from spacy_langdetect import LanguageDetector from spacy....
jill gosrani's user avatar
0 votes
3 answers
88 views

Replacing field values in a fixed-position file using awk

The ask here is to replace position 9-12 with a blank, for example, in all fixed-sized records. I used cat file | awk '{sub(substr($0,9,12)," ",$0);print}' Works, unless there is a blank ...
access_granted's user avatar
2 votes
1 answer
68 views

using awk to identify disease for patients

I have a tab seperated text file A.tsv of icd code matrix which the columns are patient id and icd codes, rows are observations for each patients. NA indicate the patient are not diagnosed as the icd ...
Ray's user avatar
  • 21
0 votes
0 answers
45 views

Easy way shortened text in EditText

I'm looking for customizable shortcut text in Android Java. I tried this: @Override public void onCreate(Bundle savedInstanceState) { String strxd = "Hello World XD"; int xdindex = ...
fkybrd's user avatar
  • 21
0 votes
0 answers
40 views

Extracting phrases and words with Apache Tika

I'm facing an issue with the extracting texts and phrases using Apache Tika. I made a unit test where it seems to be missing some expected elements and has additional unexpected elements in the actual ...
quarks's user avatar
  • 33.7k
0 votes
0 answers
22 views

Looking for a tool to search acronyms and find their alternative funny meanings

I'm searching for a tool or library that can help me search for acronyms in general and find their alternative funny meanings. For example, the tool should be able to recognize "BBC" as an ...
8ta4's user avatar
  • 11
-1 votes
1 answer
72 views

How can I use grep in Linux to find lines with multiple matching conditions in a text file?

I'm working on a Linux system and need assistance in finding specific lines within a large text file using the grep command. The criteria for these lines are: Inclusion of either 'term1' or 'term2'. ...
Bimesh Perera's user avatar
0 votes
0 answers
26 views

Tools for finding words and idioms that can be humorously misconstrued

I am looking for tools or libraries that can help me identify words and idioms that can be intentionally misconstrued to mean something else, resulting in humor. Here's an example from a video clip ...
8ta4's user avatar
  • 11
0 votes
1 answer
59 views

How to recognise the word as a single word in OCR that end of the line with a hyphen

I am using ORCmyPDF. How can I recognise end-of-the-line words that break with the hype? And remember them as one word. This will increase my search result with that words. It will make my PDF ...
Shriman Keshri's user avatar
0 votes
2 answers
54 views

Remove an item from Python list if the item's dictionary key has some value

I have a list of dictionaries. for i, d in enumerate(test_list): print(i, d) 0 {'role': None, 'content': 'Closing the gap between actual and potential crop yields', 'bounding_regions': [{'...
stackword_0's user avatar
0 votes
1 answer
37 views

Processing dataframe using multiprocess/multithread

I have a large pandas dataframe with one column "sentence" that contains text (each entry might be around 100 words and there are around 200000 entries). I would like to make a dictionary ...
Miguel Garcia's user avatar
0 votes
0 answers
39 views

Stripping Headers/Footer from Plain Text Email Content

Has anyone used/recommend any services/products/tools that have been successful at stripping header and footer content and chain replies from plain text e-mail content? I have several years of e-email ...
Stephen Hewison's user avatar
0 votes
0 answers
66 views

Hastening URL decoding process in R

I am working in R and I am trying to make the function below computationally faster. I am trying to decode UTF-8 encoded URLs. Currently, we look for the characters in the raw URL, x, that have ...
Tongo's user avatar
  • 1
-1 votes
2 answers
248 views

Automatic generation of dots in Notepad++ when pressing space

I am encountering an issue in Notepad++ where dots are automatically generated when I press the spacebar. This behavior is unexpected and causes inconvenience while working with text. To provide some ...
x Kyser's user avatar
  • 17
0 votes
0 answers
5 views

Efficient method for extracting and categorizing frequent terms from unformatted string column in SQL table

I have a string column in a SQL table that contains job titles. This field is filled in by the user and therefore has no formatting. To obtain a typology of users, I'd like to extract the terms that ...
N7Legend's user avatar
0 votes
0 answers
20 views

'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128) : BookNLP

I am trying to run BookNLP model on a txt file in Colab, but I keep getting an error. the code: input_file="alice.txt" output_directory="alice" book_id="alice" booknlp....
Troi's user avatar
  • 43
1 vote
1 answer
176 views

DJango: Textarea/Form get, process and show

Good evening, i'm currently working on a small DJango project, and am currently facing an issue: The whole of the project is to provide a textarea to the user: 1- let the user write down the text that ...
TheMailOfWork's user avatar
1 vote
1 answer
123 views

How to split text into sentences and create a new dataframe with one sentence per row?

I have a dataframe df that has 3 columns containing speechdata: filename, president, text. I split the text data into sentences using: # Split 'text' column into sentences and create a new 'sentences' ...
datascientistxxx's user avatar
0 votes
1 answer
102 views

How to combine multiple .txt files into a single dataframe?

I am trying to import 250 .txt files that are in one directory as one grouped data frame so that the dataframe consists of two columns with one column taking on the .txt file name and one column ...
datascientistxxx's user avatar
0 votes
2 answers
62 views

Replacing a string unless it contains a given substring with sed

I have a file which contains many links of the form <a href="foo">. These need to be given a .html extension, and I have the following command to do so: sed -i 's/\(href="\)\([^&...
tsvallender's user avatar
  • 2,669
2 votes
3 answers
84 views

Want to change prefix characters, then add suffixed characters in changed string

I want to create a poor-man's replace for HTML-izing a string text. A sample text is: THIS IS A TEST.@@01 THIS IS ANOTHER TEST.@@02 What I have been using (which works for both Mac and LINUX) is this:...
infracritical's user avatar
0 votes
1 answer
130 views

Standardize the text in the pandas column with some common string

I have following DataFrame df id1 id2 text_column key1 220 ABC corp key1 220 ABC Pvt Ltd key2 300 PQR Ltd key2 300 PQR key2 300 PQR something else key2 ...
TLanni's user avatar
  • 330
2 votes
1 answer
115 views

how to get scikit-learn's tokenizer to use punctuation to separate tokens?

I have the following code using scikit-learn to count ngram frequencies: c = ["data. format", "data are format hello world"] vectorizer = CountVectorizer(ngram_range=(1,2)) X = ...
user20c's user avatar
  • 31
-2 votes
1 answer
123 views

Removing html code from text data in Spark

i'm working with Stack Exchange data dumps and I want to clean body of a question from code which occurs really often. I tried using BeautifulSoup but it leaves the text which occurs inside the code ...
Katy's user avatar
  • 63
-1 votes
1 answer
80 views

How to keep blank lines while reading text file Python

Reading in a generic text file with blank lines, i.e. no tabs, no spaces along some lines. I want to use these blank lines as a delimiter to process the files in the program, but I cannot stop them ...
MinimalCodingIQ's user avatar
0 votes
1 answer
50 views

How to split words in a paragraph when there is a <a> tag with href attribute within the <p> tag

I am working on a web scraping project and I want to split the words in a paragraph when there is an <a> tag with href attribute within the <p> tag. I'm new to scraping and programming in ...
guy atia's user avatar
0 votes
0 answers
107 views

How to properly split streaming text into paragraphs?

I am using okhttp subscribing to a SSE( https://en.wikipedia.org/wiki/Server-sent_events) stream, the onEvent callback is use to receive new data from upstream and passed by data param as String It ...
djy's user avatar
  • 737
-3 votes
1 answer
102 views

Python : remove duplicate character

string input: Tem1 = 'Hhelloo ookkee' I want to make output like Tem1 = 'helo oke' I have try this link form stackoverflow (Python: Best Way to remove duplicate character from string) I've tried ...
0nespo's user avatar
  • 1
1 vote
1 answer
42 views

Pull a value from a text file to use as a comparitor

I need the pull the temperature from a text file, and then use the value as a comparitor to run one or the other python script. the data looks like this: time=14:10 temp=22.4 <----- I need this ...
BobW's user avatar
  • 37
0 votes
3 answers
42 views

find values that occurs the least in 2 columns file

I've a dynamically generated txt file of this structure -> Code[space]Link This is a simple 15210 https://test1.com 4 https://test2.com 1020 https://testc.com 152 https://testz.com 152 https://...
nora's user avatar
  • 23

1
2 3 4 5
40