Questions tagged [nlp]
Natural language processing (NLP) is a subfield of artificial intelligence that involves transforming or extracting useful information from natural language data. Methods include machine-learning and rule-based approaches.
20,376
questions
-1
votes
0
answers
6
views
How to deploy llama2 using free aws or azure vms
I know llm (transformer) basics and architecture.
I want practical understanding of openai and llama2 training or finetuning for specific dataset, inference.
Can I explore this with any freely ...
0
votes
0
answers
9
views
set up authentication flow correctly in Flask app
I'm working on integrating OAuth 2.0 into my Python Flask application, and I'm encountering some issues. I've followed the documentation for the OAuth provider I'm using, but I'm not sure if I've set ...
1
vote
1
answer
20
views
Pretrained model with stride doesn’t predict long text
My objective is to annotate long documents with bioformer-8L. I have been said to use stride and truncation so I don’t have to split my documents in chunks of 512 tokens.
In the training phase, I ...
0
votes
0
answers
14
views
What are the pros, cons, development and future for GPT, Bard and LLAMA?
As data scientists, navigating the evolving landscape of development in the wake of LLM's impact is akin to embarking on a thrilling quest. How do we chart our course forward in this dynamic realm? ...
-1
votes
0
answers
13
views
OpenAi to retrieve data from my application
I manage a CRM application with thousands of opportunities, accounts, contacts, and more. I'm looking to implement a chat-like feature allowing users to ask questions and retrieve data from the stored ...
0
votes
0
answers
34
views
Issue with DPO training on OpenLLAMA 3B V2 model: "I am sorry I cannot reply to that question" response during inference
Description:
I am currently working on a project involving Direct Preference Optimization (DPO) training with the TRL library from Hugging Face. The goal is to align responses from the OpenLLAMA 3B V2 ...
0
votes
0
answers
17
views
Why building a new scorer outputs an empty string for deepspeech 0.9.3
I am trying to create a limited corpus and train a language model to use for a deepspeech scorer.
I have followed the information provided in the docs here
I read a helpful guide posted for an older ...
2
votes
0
answers
24
views
Incremental Inverse Document Frequency without storing the past information
I compute the tf-idf everyday in my pipeline using pyspark to evaluate the significance of a keyword in a specific document. This enables me to generate a summary for utilization in my machine ...
0
votes
1
answer
19
views
Memory-efficient BERT Text Embedding for Large Dataset Preprocessing in TensorFlow
I'm working with a dataset containing approximately 920,614 rows and multiple columns, including "orig_item_title," "sub_item_title," "is_brand_same," and "...
0
votes
0
answers
16
views
GPE using SpaCy identifies locations which are not geographical locations
I am using spacy library in my NLP code to identify the prominent geographical locations mentioned in the text.
But Spacy is identifying words which are not geographical locations along with the ...
0
votes
0
answers
7
views
Pretraining BERT Models from scratch vs Further Pretraining
I want to pretrain an Arabic BERT model on domain-specific data to make it suitable for a specific domain problem, which is the classification of citizen reviews about government services into ...
0
votes
1
answer
27
views
Using langchain and LLaMA2 to QA with a large SQL DB
I was working with a sqlite DB that I created for a large dataset (~150k rows).
Code snippets:
db = SQLDatabase.from_uri("sqlite:///MLdata.sqlite")
SQLITE_PROMPT_TEXT = '''You are a SQLite ...
0
votes
1
answer
21
views
How do you save the model every time there is a better loss value achieved during training while using Trainer from HuggingFace?
Searched a lot but there isn't a single solution which can make us save the weights of "every" loss decrease. I'm aware that I can get the "best loss" model in the end and it'll be ...
-1
votes
1
answer
27
views
Bert topic clasiffying over a quarter of documents in outlier topic -1
I am running Bert topic with default options
import pandas as pd
from sentence_transformers import SentenceTransformer
import time
import pickle
from bertopic import BERTopic
llm_mod = "all-...
0
votes
1
answer
32
views
Modifying the sentiment of certain words in tidytext get_sentiments()
I am trying to modify the sentiment of a few specific words in my df to make them more suitable for my context, where they were used with a negative connotation but have been classified as having a ...
-1
votes
0
answers
44
views
How to Implement Content-Based Filtering for Voice Search?
I need to implement a content-based filtering "voice search" process on iOS using Swift language with Natural Language Processing (NLP).
The primary goal is to extract specific content or ...
0
votes
2
answers
38
views
spaCy displacy output using anvil.works server
I am attempting to display entities using spaCy's displacy feature.
The output of my render is being shown in my Jupyter Notebook code cell with my anvil.server.wait_forever() code.
Here is an example ...
-3
votes
0
answers
35
views
High number of false positives using TF-IDF method [closed]
I am working on an arabic book with back-of-the book indexes and I want to preprocess this book and apply TF-IDF to get automatically back-of-the book indexes but my problem is that when analysing I ...
-3
votes
0
answers
34
views
How do I prevent overfitting and get a higher accuracy? [closed]
I am doing aspect based sentiment analysis and currently my model when fitting starts with the validation loss and accuracy higher than the training, but then it flips and training performs a lot ...
-2
votes
0
answers
33
views
How do I map text descriptions to categories? [closed]
I am attempting to map text descriptions to categories. I am aiming to train a model using a list of descriptions to category mappings, and then make a prediction using any input text. A valid output ...
-1
votes
0
answers
24
views
How to align AMR edge labels with PropBank Role Names
So I'm trying to use the Penman library (and possibly some additional ones) to transform an AMR to use the human-readable PropBank role name instead of generic ARG0, ARG1 etc. So for example:
(w / ...
-2
votes
0
answers
23
views
Handling 'None' values in SynTagRus dataset from Universal Dependencies library for NLP task
I am working on a Natural Language Processing (NLP) task with the Russian language using the SynTagRus dataset from the Universal Dependencies library. I load the dataset as follows:
train_dataset = ...
-1
votes
0
answers
32
views
New error everyday,Error occuring in Jupyter notebook
It was working a week ago...
!pip install torch transformers bitsandbytes accelerate sqlparse
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "codellama/...
-1
votes
0
answers
41
views
How can I use FEEL to analyse sentiments of french political tweets in R? [closed]
I have a corpus of french political tweets that I wish to analyse. I just imported FEEL (French expanded emotion lexicon) dataset with 14,000 french words into R that looks like follows (the words are ...
-2
votes
0
answers
7
views
ValueError: Weights for model 'sequential_2' have not yet been created. Weights are created when the model is first called on inputs"
I'm trying to practice machine translation using Transformer following deep learning book. When I build Transformer model getting this error "ValueError: Weights for model 'sequential_2' have not ...
-1
votes
0
answers
13
views
Predicting Tweet User engagement with BERT
I can predict user engagement given the tweet contents and the time it was posted. My initial approach uses a regression model on the BERT embeddings for the tweet contents. However, the time the ...
0
votes
0
answers
46
views
how to do text anonymization using Hugging Face transformers?
I just followed this guide https://medium.com/@luccailliau/text-anonymisation-using-hugging-face-transformers-75b5d7392833 but the code doesn't work and returns
TypeError: 'BatchEncoding' object is ...
1
vote
1
answer
36
views
X has 95812 features, but RandomForestClassifier is expecting 178341 features as input [duplicate]
I'm having a Random Forest model using text data. However, when I try the model on new data (test set), the number of features are not compatible between the train and test set. Also, the transform on ...
0
votes
1
answer
21
views
Fine Tuning a T5 Model on Squad
Goal: Fine Tune T5 on SQUAD dataset for extractive question and answer
Question: If my tokenized dataset contains input_ids, attention_mask, targets_ids, and target_attention_mask, why is the trainer ...
0
votes
1
answer
53
views
NLP preprocessing text in Data Frame, what is the correct order?
I’m trying to preprocess a data frame with two columns. Each cell contains a string, called "title" and "body".
Based on this article I tried to reproduce the preprocessing. ...
0
votes
0
answers
23
views
Getting Logits and Labels mismatch
I'm trying to build a chatbot using the tutorial text
The dataset the tutorial uses is no longer available so I did my own with chatgpt.
A snippet of the dataset:
How was your day? Amazing! Thanks for ...
1
vote
1
answer
12
views
How to customize the number of encoders/decoders in a pre-trained transformer
I am implementing a pretrained transformer model using Python's transformer module to perform text summarization and I would like to compare the performance of the fine-tuned BART transformer given ...
0
votes
1
answer
18
views
Huggingface QA model results decoding
I'm trying work with a transformer model (for the question answering task). I am submitting a short string as an input example.
It's my simplest code:
from transformers import AutoTokenizer, ...
0
votes
0
answers
8
views
Issues with Sokuon Conversion in pykakasi Library for Japanese to Romaji Translation in Python
I am attempting to use the pykakasi library in Python to convert Japanese text to Romaji. However, I am encountering issues with the conversion of sokuon (促音). Here is the code I am using:
import ...
-1
votes
0
answers
72
views
NER (Named-entity recognition) with spaCy in XML file
WHAT I HAVE
I have a XML file:
(This is a dummy example for the purpose of the question)
<?xml version="1.0" encoding="UTF-8"?>
<document>
<book id="bk101&...
-3
votes
0
answers
19
views
creating a semantic search model [closed]
I am working on a project to create an application that allows users to query cricket data from the past 10 years. I plan to compile information on matches, players, and news in a text file. My goal ...
0
votes
1
answer
23
views
Understanding output_attentions
I have a question about output_attentions, and I need to make a heatmap about the attention of the final layer of the BERT model. But I do not know if the output_attentions[0] is the first or last ...
1
vote
1
answer
41
views
How to build a model and train it with tensorflow keras sub classing
I have written a custom Encoder and Decoder layers that implements the architecture described in the Attention Is All You Need paper. Everything works fine until I trying compiling it, I get one error....
0
votes
1
answer
25
views
fine-tune BERT for semantic textual similarity
I’m trying to use BERT (or any language embedding models) to solve a semantic text similarity problem: given a product A, find product B, which is basically the same underlying product, with a few key ...
0
votes
1
answer
52
views
Fine-tuning Hugging Face’s Llama Model with Unlabelled Data from PDFs from certain domain
I’m unsure about the next steps. Specifically, I have the following questions:
How can I prepare my unlabelled data for the fine-tuning process?
What’s the best way to fine-tune the Llama model with ...
1
vote
1
answer
39
views
Importing deepspeech error: Module not found
I am trying to install DeepSearch library so that I can use the pretrained model to build the Speech to Text Project.
ModuleNotFoundError Traceback (most recent call last)
<...
0
votes
1
answer
22
views
Clarification on Padding Process in BERT Model Construction
In my endeavor to construct a BERT model from the ground up for the purpose of gaining hands-on experience and a comprehensive understanding of the model, I have encountered a point of confusion ...
0
votes
1
answer
21
views
How to Integrate Custom OpenAIModel into a AutoModelForSequenceClassification Model?
I've developed a custom OpenAIModel module that acts like BERT models but makes an OpenAI embeddings request and returns the results when called. I want to use this module, utilizing Hugging Face's ...
0
votes
1
answer
36
views
Does there exist a good generic production-level fine-tuning script for transformer models?
Does there exist a generic script for fine-tuning generative language (transformer) models?
It could be based on a library that supports loading many different models (for example, HuggingFace).
I can ...
0
votes
0
answers
26
views
I'm getting "None" in the config path, when trying to download the model, from CoquiTTS
I'm trying this piece of code:
from TTS.api import TTS
from TTS.utils.manage import ModelManager
from TTS.utils.synthesizer import Synthesizer
path = "C:/Users/sanca/AppData/Local/Programs/...
-3
votes
0
answers
31
views
Rewrite a sentence Using GPT-2 [closed]
I am seeking to prompt GPT-2 to generate a sentence starting with 'The actor is ...'. Initially, GPT-2 generates
'The actor and his wife allegedly met with Jeev before the film was released in May ...
0
votes
0
answers
20
views
Matching error patterns using spacy matcher issue. Giving a match where it is not supposed to
Iam working on finding a error pattern using spacy matcher in python (from spacy.matcher import Matcher)
"The manufacturer recommends\nthe wall is painted" - should match the pattern
"...
0
votes
1
answer
31
views
Clearing context window of LLM in Huggingface
I want to use inference to ask different questions to LLMs taken from huggingface. But, I want to ask the prompts without the model having info about the previous prompts. Does the model automatically ...
0
votes
0
answers
18
views
Stanford CoreNLP library doesn't tokenize new lines
I'm trying to perform tokenization operation with the following code:
private String[] nlpProperties = {"annotators","tokenize","tokenize.verbose","true","...
-2
votes
0
answers
22
views
Choosing Between Rule-Based Parsing and NER for Resume Screening in Python
I know this is fundamental, but I just can't wrap my head around some of the ML/Ai use case's. I can understand about predicting or classifying a certain element or generating new text but use case's ...