Skip to main content
Filter by
Sorted by
Tagged with
-1 votes
1 answer
37 views

Automatic Word Boundary Detection for German

I want to rephrase that: I need a corpus of German words so that I can check if a segment is a word. My solution so far is to take the string, check if it's in the dictionary and if not, delete the ...
kiwi-123's user avatar
0 votes
0 answers
38 views

Can log2 be substituted with ln in logDice association measure in R?

I have a question about logDice association measure for collocational analysis. This is the formula for logDice: logDice = 14 + log2(2.w1w2/w1+w2) Where: w1w2 = the frequency of the word x and y w1 = ...
pindakazen's user avatar
0 votes
0 answers
20 views

Using numeric column in dataframe within formula in R

I am a newbie and trying to do a transformation from one unit to another using the numbers that make up a column in a dataframe that I'm using. The formula is [(26.81x)/(1960+x)]-0.53. I've tried a ...
HaTo's user avatar
  • 1
1 vote
0 answers
66 views

Query Wikidata via SPARQL to get specific word etymology from Wiktionary

I'm trying to get the specific word etymology from Wikidata. For example, this query to get the word "exact" in Wikidata but I wasn't able to get the etymology part for this word. SELECT ...
Omar Al-Howeiti's user avatar
-1 votes
1 answer
650 views

What does "assign A to B" mean?

If I say "assign A to B", does it mean (a) A ← B or (b) B ← A? In other words, is it (a) A or (b) B that is being modified? (a) makes sense because A has responsibility over B, so A is ...
glibg10b's user avatar
  • 379
0 votes
1 answer
71 views

Problems with reproducing the training of the spaCy pipeline

I'm trying to reproduce the training of one of the spaCy pipeline for Italian language: it_core_news_sm. This pipeline is trained on 2 datasets: UD_Italian-ISDT for the conllu tasks WikiNer for NET ...
Andrea Lavista's user avatar
1 vote
0 answers
117 views

In NLTK, how to generate a sample of sentences from PCFG, respecting the probabilities

NLTK has a generate method which enumerates sentences for a given CFG. It also has a PCFG class for probabilistic context-free grammars. Is it possible generate a sample of sentences with respect to ...
Albert Gevorgyan's user avatar
0 votes
1 answer
212 views

Weighted Distance Matrix for QWERTZ Keyboard for Levenshtein Distance Algorithm

I have a weight Matrix for a Levenshtein Distance Algorithm which looks like this int[,] weights = new int[6, 6] { { 0, 1, 2, 1, 1, 2 }, { 1, 0, 1, 2, 1, 2 }, { 2, 1, 0,...
Marco-rm-f's user avatar
0 votes
1 answer
334 views

How to develop a corpus(corpus analysis)

I am goingt to build a linguistic corpus, but i don't understand which technologies should i use for it. Is it true, that for developing a courpus for any language i necessarily have to use IMS Corpus ...
Murad Mammadzada's user avatar
-1 votes
1 answer
66 views

How do I study linguistic features of NLP libraries like spacy/NLTK in-depth? [closed]

My Objective I am trying to do Natural Language Processing(NLU to be precise) with spaCy. But the more I dive into this wonderful library, the more I realize that I need to understand some linguistics ...
Esa Anjum's user avatar
0 votes
1 answer
96 views

Tool for detecting differences between text passages from two different groups

I have text data from two different groups. In total I have around 4000 text passages with around 300 words. I am searching for a tool that allows me to analyze the difference between these two groups....
Irazall's user avatar
  • 157
0 votes
0 answers
26 views

R - readtext and list of .xml files

I'm trying to create a corpus and a vcorpus with a bulk of .xml files, for quantitative linguistics With txt files I usually write library(tm) library(stopwords) library(magrittr) library(dplyr) ...
 SubotnikOne's user avatar
2 votes
0 answers
56 views

How can I determine if a word is a part of an english word or is a portmanteau (a word created by combining parts of valid English words)?

I am trying to create a validator that takes in words and tries to determine if the word is one of the following: It is a valid English word It is a part of an English word It is an abbreviation It ...
user3188603's user avatar
0 votes
0 answers
42 views

Reducing time for minimum edit distance in python?

I am trying to create a list with the edit distances between each word in a set of documents, ranging from 10k-42k words. If my idea of edit distance is correct, I would end up with a distance for ...
dmoses's user avatar
  • 15
1 vote
0 answers
156 views

Customization of Wav2Vec2CTCTokenizer with rules

my goal is to fine-tune an ASR model, WavLM, that relies on the pretrained tokenizer Wav2Vec2CTCTokenizer. I want to fine-tune this ASR model with another language and to perform the tokenization ...
Sara Picciau's user avatar
1 vote
0 answers
289 views

spaCy custom tokenizer to separate word with underscore and also to include the whole word

After referring to the link: How to tokenize word with hyphen in Spacy I got to know how to tokenize by separating words containing hyphen/underscore but my requirement is to tokenize by separating it ...
Shakkir Moulana's user avatar
1 vote
1 answer
1k views

How to do the post hoc test in the linear mixed model if I have three predictors (two factor variables and one numeric variable)

I'm using a linear mixed effects model to analyze the reaction time of learners of English as a second language. I have two factor variables - grammaticality (grammatical v.s. ungrammatical) and ...
Yang Cao's user avatar
2 votes
2 answers
200 views

Can I use a vector as a regex pattern parameter in R?

I want to search a phonetic dictionary (tsv with two columns, one for words, another for phonetic transcription: IPA) for certain consonant clusters according to the type combination (e.g. fricative+...
marcocamilo's user avatar
1 vote
0 answers
54 views

How do I recognize (spaceless) English text?

[Note. This question has mild spoilers, but no answers, for a crossword puzzle in the May 2022 issue of The MagPie.] In the puzzle, we are given a 52-character string. It is encrypted using Playfair,...
Frank Yellin's user avatar
  • 10.8k
1 vote
1 answer
61 views

How to implement the theory of copy movement in Prolog?

Using this input: parse(Parse, [what,did,thomas,eat], []) I want to produce this output: sbarq( whnp( wp(what) ), sq( vbd(did), np(nnp(thomas)), vp( vb(eat), whnp( ...
madhatter's user avatar
1 vote
1 answer
478 views

How to turn an adjective into a noun in python (pymorphy2)?

I have a word in Russian (which is actually where the difficulty arises). It's an adjective. And I need to make it in a noun form. I found an interesting library that can parse words, inflect and ...
kostya ivanov's user avatar
0 votes
1 answer
40 views

Algorithm to rank the simplicity of a random name

I have been looking for a name for a new project. I want the name to have available domains and social media handles. For months, all those I can think of are taken. So I generated a list of names ...
emonigma's user avatar
  • 4,196
-2 votes
1 answer
188 views

Tips for writing an algorithm for paraphrasing sentences(machine learning)

I am doing a project at the university and I need to train an algorithm to rephrase sentences, what can you advise for implementation? Is it possible to use a translator to translate into another ...
Zaytsev Alexandr's user avatar
1 vote
2 answers
731 views

Relationship between vocab size and complexity

I have 2 corpuses, if one has a larger vocabulary size than the other, does it mean its language is more complex? Apart from complexity of the language, what else can effect the size of the vocabulary ...
daniel's user avatar
  • 71
0 votes
1 answer
248 views

Draw an FSA that recognizes: (A∗ | AB+). (The bar outscopes the other operators, so its equal to: (A∗) | (AB+).) Use as few states possible

I've attached what I have. My problem is that I don't know if its correct and if I've even used the fewest states possible to answer this question. Really appreciate any help on what I currently did ...
theworkforceof's user avatar
0 votes
0 answers
52 views

Generating pronounceable digrams in Python

I am generating all two letter (digram) combinations of the standard alphabet import string import itertools UPPERCASE_ALPHABET = string.ascii_uppercase TWO_LETTER_PERMS = itertools.permutations(...
Robert Alexander's user avatar
0 votes
1 answer
138 views

Chinese Whispers for NLP How to implement for my corpus file

I'm new to Python, I want to make this code to implement my corpus from (.csv)files Can anybody help me how to implement it like in the picture below. from chinese_whispers import __version__ as ...
elle's user avatar
  • 7
1 vote
2 answers
461 views

Sentence tokenization w/o relying on punctuations and capitalizations

Is there a possible approach for extracting sentences from paragraphs / sentence tokenization for paragraphs that doesn't have any punctuations and/or all lowercased? We have a specific need for being ...
ZZZZZZZZZ's user avatar
  • 199
0 votes
1 answer
73 views

Can Mandarin pronunciation be coded by ARPABET phone set?

The code doesn't need to be exactly correct. A pronunciation in English can be used to represent a similar pronunciation in Chinese, e.g. /ʈ͡ʂ/ can use "CH" to represent.
CharlieLei's user avatar
1 vote
2 answers
639 views

Python - How to loop through each index position in a list?

Given a list [[["source1"], ["target1"], ["alignment1"]], ["source2"], ["target2"], ["alignment2"]], ...] , I want to extract the words in ...
c_metaphorique's user avatar
2 votes
0 answers
49 views

Entropy Calculation as a Linguist Feature

I was going through a paper on Sarcasm Detection which proposed a Hybrid Attention Based LSTM. The Model included some linguistic features to be extracted first, which included Entropy of a word in a ...
ragingmartyr's user avatar
0 votes
1 answer
265 views

What kind of encoding is this html encoding?

I am doing a project that involves searching words in the Arabic script on Wiktionary, and when I do a GET request on certain word pages, I get something like this for example: title="\xd8\xb1\...
AdmosJagashvili's user avatar
0 votes
0 answers
56 views

Identifying phrases which contrast two corpora

I would like to identify compound phrases in one corpus (e.g. (w_1, w_2) in Corpus 1) which not only appear significantly more often than their constituents (e.g. (w_1),(w_2) in Corpus 1) within the ...
MRicci's user avatar
  • 143
0 votes
1 answer
400 views

Convert lower case letter 'ß' to upper case letter 'ẞ' in PHP [duplicate]

I'm trying to convert the lower case letter 'ß' to the upper case pendant. I already tried using the built-in functions strtoupper and mb_strtoupper. The following happened when I use strtoupper: echo ...
Tarik Weiss's user avatar
1 vote
0 answers
198 views

Google Natural Language Syntax Analysis

I'm not sure this is a coding issue/question. I'm using Google's NLP to analyze the syntax of some sentences and I'm seeing some inconsistencies with Plural vs Singular designation. Perhaps I'm doing ...
Mcorv's user avatar
  • 161
1 vote
0 answers
88 views

NLTK.sem.logic cannot infer logical types of functions given types of argument and result, why?

I am working with NLTK.logic.sem to handle the logical structure of sentences. I also use NLTK to handle inferring the logical types of expressions. The NLTK book has an example of when it cannot ...
L. Suurmeijer's user avatar
1 vote
2 answers
941 views

Find a word in a list of words that has minimum distance with a given word

let say given a word list ['windows','hello','python','world','software','desk'] and an input word 'widow', how to (quickly) find the word from the word list that has the minimum edit distance with ...
Leo's user avatar
  • 65
0 votes
1 answer
63 views

Removing from a text sequences of items from a list

I'm conducting research on code-switching. I have a collection of bilingual Polish text messages with English code-switches (say, corpus A) as well as an English dictionary (also a list, corpus B). I ...
Joanneboron's user avatar
3 votes
3 answers
844 views

PROLOG - Get a list of all the rules an entity verifies

I'm formalizing linguistic data into predicates and entities and doing some reasoning in prolog. Imagine I begin with: breathe(X) :- snore(X). sleep(X) :- snore(X). rest(X) :- sleep(X). live(X) :- ...
EndingBeginningCycles's user avatar
1 vote
3 answers
96 views

Negative lookbehind in RegEx: Matching multiple POS-tags at once

I am still fairly new to regex, so I would appreciate any help. I am trying to use regular expressions to find specific grammatical patterns in a text corpus that was part-of-speech-tagged using the ...
Serendipity's user avatar
0 votes
0 answers
128 views

NLTK - Access Brown corpus as one file

I have recently started working with NLTK, and I have downloaded the Brown corpus to do analyze its data. However, I see that the corpus is split into around 500 files. I need to run a script on the ...
The Philgrim's user avatar
1 vote
1 answer
215 views

How to determine structure of answer for a WH-question

Consider a wh-question such as "Who closed the door?". Personally, I can determine that an answer will look like "NP closed the door.", where NP would be a noun phrase. Another ...
DebNatkh's user avatar
0 votes
1 answer
727 views

Why mono sounds are preferred over stereo sounds when it comes to analyse acoustic parameters like intensity, RMS amplitude, fundamental frequency etc

In phonetic analyses, while using software like Praat, I have seen people's liking mono sounds over stereo sounds. Even if the recordings were done using a stereo microphone, the audio files are often ...
Pranav_b's user avatar
4 votes
1 answer
2k views

Distance between strings by similarity of sound

Is the a quantitative descriptor of similarity between two words based on how they sound/are pronounced, analogous to Levenshtein distance? I know soundex gives same id to similar sounding words, but ...
Al Guy's user avatar
  • 201
0 votes
2 answers
309 views

Finding the rhyming words in a corpus with R, regex

I have the following corpus: corpus_rhyme <- c("helter-skelter", "lovey-dovey", "riff-raff", "hunter-gatherer", "day-to-day", &...
Viola Green's user avatar
0 votes
2 answers
71 views

Tartar language and UTF-8

I'm working on a project that includes transforming latin symbols into the corresponding cyrillic ones. I'm talking about Tartar language which is used by one of many nations in Russia. I tried find ...
Rodj's user avatar
  • 3
1 vote
0 answers
30 views

What is the more natural parsing, the one that leads to the preferred reading of the sentence

I have those rules: and those two possible parse trees: I am asked for the next question: What is the more natural parsing, the one that leads to the preferred reading of the sentence? Can anyone ...
Ilya.K.'s user avatar
  • 321
0 votes
1 answer
25 views

Get rows from MySQL table that have a column in common but slightly diffrent value [duplicate]

Might not even be a programming question, but the problem I'm having is I have a big MySQL table with products from different sources (stores) and in many cases it's the same product with different ...
himmip's user avatar
  • 1,728
3 votes
1 answer
367 views

Python regex selection of verbs with present perfect

In a given string, I am trying to catch verbs that are in present pefect tense. I do that by using the following regular expression in python: import re sentence = "The Batman has never shown his ...
krasnapolsky's user avatar
0 votes
1 answer
67 views

To remove non- pair entries in a dataframe?

A column 'Consonant' in my dataframe 'df' has two levels with numbers, 250 for 'Geminate and 214 for 'Singleton'. Because Geminates are more in number, how can I remove the extra entries for Geminates,...
Pranav_b's user avatar

1
2 3 4 5
7