Questions tagged [gpt-2]
Use this tag with Generative Pre-trained Transformer 2 (GPT-2). Do not use with GPT-3 or the ad tagging library (GPT).
202
questions
0
votes
1
answer
23
views
How use past_key_values in pytorch for caching
why using past_key_values of "hello, my dog is" plus the input id of "cute" doesn't output the same as using "hello, my dog is cute"
In my understanding the ...
0
votes
0
answers
49
views
The maximum value of input_ids (50257) must be smaller than the embedding layer's input dimension (50257)
I am trying to create a categorical model that produces one of the three result for some data input: True, False, Invalid.
as mentioned in the title I get this error when I try to evaluate the model, ...
0
votes
1
answer
35
views
Why can we set LLM's input and output to be the same when fine tuning on text generation task?
I'm trying to fine tune my GPT-2 model for song lyrics text generation, and I have a couple of song lyrics on hand. However, I'm confused about how to fine tune GPT-2 model that doesn't have standard ...
3
votes
0
answers
18
views
compute LM conditional probability of a text given a prefix
I am seeking a pythonic way to compute the probability of text chunk condition on a text prefix. Example below:
from transformers import GPT2Tokenizer, GPT2LMHeadModel
tokenizer = GPT2Tokenizer....
1
vote
0
answers
14
views
Perplexity metric for GPT2 model is lower for non-English text
I am currently working on a project of calculating perplexities of various causal LLMs for different languages to estimate their behaviour if there is an input in a form of the language, that ...
0
votes
0
answers
69
views
GPT2 and HuggingFace Transformers: repeating user_input in response
I am just getting into making my first model using GPT 2 and the HuggingFace Transformers library in Python as my gateway into understanding model training. I'm running this on Python 3.7 with the ...
0
votes
0
answers
24
views
Softmax output and probabilities not matching up?
I'm trying to test how well a GPT model can classify verbs according to the left-side context in a given input sentence with a masked term.
For example,
Input sentence:
"The ballerinas' costumes ...
0
votes
1
answer
39
views
Why is my streamlit app not correctly summarizing my mp3-transcription?
I'm working on a Streamlit app that processes MP3 files. The main steps include:
Uploading an MP3 file.
Splitting the audio into smaller chunks using pydub.
Transcribing these chunks using OpenAI.
...
0
votes
1
answer
117
views
How do fix GPT2 Tokenizer error in Langchain map_reduce (LLama2)?
I'm using AWS Sagemaker Jumpstart model for Llama2 13b: meta-textgeneration-llama-2-13b-f
On running a Langchain summarize chain with chain_type="map_reduce" I get the below error. I do not ...
0
votes
0
answers
19
views
ValueError: Expected input batch_size (1052) to match target batch_size (508) when fine tuning GPT 2 model
Hello there I'm attempting to train a GPT 2 model how to summarize passages without compromising their emotional impact. Consider summarizing a chapter from a book, but we want the reader to ...
1
vote
0
answers
191
views
Invalid key: 409862 is out of bounds for size 0
How I can fix this:
I writed code for training GPT-2 on dataset by Hugging Face, but I have an error and don't know why I got this error:
---------------------------------------------------------------...
0
votes
0
answers
34
views
'utf-8' codec can't decode byte 0xc3 error when using tensorflow.keras.layers import TextVectorization
I am trying to execute the steps given in a blog post (https://stackabuse.com/gpt-style-text-generation-in-python-with-tensorflowkeras/) but getting the error in the below block:
vectorize_layer.adapt(...
0
votes
0
answers
66
views
GPT-2 PyTorch Custom Training
I have this implementation of OpenAI GPT-2 LLM in PyTorch. Can you please help me write a training loop for it, if my dataset looks like this text file:
<|user|>How are you?<|bot|>I’m fine&...
0
votes
0
answers
18
views
tf.compat.v1.estimator.Estimator(): NameError: name 'model_fn' is not defined.Getting errors in add_argument as well. Not recognizing paths mentioned
I am trying to create a pet LLM using GPT-2 following instructions here: https://thomascherickal.medium.com/how-to-create-your-own-llm-model-2598615a039a
The code gives syntax error while calling tf....
0
votes
0
answers
54
views
Title: Generating Sentences with TRL while Maintaining Sentiment - Issue with "AutoModelForCausalLMWithValueHead"
I am currently working on generating sentences with TRL (Transformers Reinforcement Learning) while preserving the same sentiment as the sample sentences. However, I've come across an issue with the ...
0
votes
1
answer
203
views
Hugging Face Inference API returning short generated text with GPT-2 model
I'm using the Hugging Face API with a GPT-2 model to generate text based on a prompt. However, I'm encountering an issue where the generated text is consistently too short, even though I'm specifying ...
0
votes
1
answer
569
views
How to generate text using GPT2 model with Huggingface transformers?
I wanted to use GPT2Tokenizer, AutoModelForCausalLM for generating (rewriting) sample text. I have tried transformers==4.10.0, transformers==4.30.2 and --upgrade git+https://github.com/huggingface/...
1
vote
0
answers
69
views
GPT2 LLM fine-tuned model not generating expected answer
I am finetuning gpt2 model to answer questions with given faq.json.
There is some issue with the answer generated by below code. I am assuming I have not done encoding/decoding of questions and ...
0
votes
0
answers
44
views
Non-meaningful response from finetuned GPT-2 model
I am experimenting with the abilities of GPT-2 for question answering aiming at making a good task-based chatbot. I trained my model on the air_dialogue dataset from huggingface https://huggingface....
0
votes
0
answers
22
views
FineTune GPT2 on Insurance Domain data
I am new to LLM and I am trying to finetune GPT2 from Huggingface on Insurance domain data. I am not getting results from the Trained data instead I am getting different results.
My Training Data is a ...
2
votes
1
answer
52
views
How to refine a trained model in gpt2?
Im currently trying to work on text generation with my own text. I have trained my model with gpt2 with my own text. But it is giving random answers. For some questions it is giving me relevant ...
0
votes
0
answers
147
views
How to train gpt2 model to learn from the training text I have given?
I'm trying to train and fine tune my gpt2 model with my own sample training document. I'm using the code similar to this: https://www.kaggle.com/code/changyeop/how-to-fine-tune-gpt-2-for-beginners . ...
2
votes
1
answer
4k
views
What is the cause of HFValidationError in this code and how do I resolve this error?
My python code in Chaquopy android studio Project:
import torch as tc
from transformers import GPT2Tokenizer, GPT2Model
def generate_text(txt):
"""
Generate chat
https://...
0
votes
1
answer
128
views
Unsupervised fine-tuning on custom documents after the supervised fine tuning on general question-answers dataset. Will it be useful for GPT-2 model?
I know the formal way of training a GPT2 model on custom documents is to first do semi-supervised fine tuning on the text of the documents followed by supervised fine-tuning on question answers from ...
0
votes
0
answers
60
views
tf.compat.v1.estimator.Estimator(): NameError: name 'model_fn' is not defined
I am trying to create a pet LLM using GPT-2 following instructions here: https://thomascherickal.medium.com/how-to-create-your-own-llm-model-2598615a039a
The code gives syntax error while calling tf....
0
votes
0
answers
47
views
TypeError: argmax(): argument 'input' (position 1) must be Tensor, not numpy.ndarray
I am traning a model GPT-2 using my curated dataset, and getting the following error. When I am trying to debug any issue, a new error comes. Can anyone helpme to fix my script so that it can run. The ...
6
votes
2
answers
962
views
Generate the probabilities of all the next possible word for a given text
i have the following code
import transformers
from transformers import pipeline
# Load the language model pipeline
model = pipeline("text-generation", model="gpt2")
# Input ...
1
vote
0
answers
67
views
Expected input batch_size (28) to match target batch_size (456), Changing batch size increase the target batch size with GPT2 model
I was practising fine-tuning a gpt2 model on a simple question-answer dataset when I encountered this error. I have studied other answers, but my input dataset shapes look fine.
def tokenize_data(...
0
votes
1
answer
229
views
IndexError: index out of range in self while using GPT2LMHeadModel.from_pretrained("gpt2")
I am working on this question answering code and using pretrained GPT2LMHeadModel. But after tokenization when I pass the inputs and attention mask to the model it is giving index error. My code:
...
0
votes
0
answers
307
views
TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]] when training GPT-2
I'm learning how to train generative models using the transformers library from HuggingFace, however, I keep having this error: TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[...
0
votes
1
answer
172
views
My gpt2 code generates a few correct words and then goes into a loop of generating the same sequence again and again
The following gpt2 code for sentence completion generates a few good sentences and then ends in a loop of repetitive sentences.
from transformers import GPT2LMHeadModel, GPT2Tokenizer ...
0
votes
0
answers
171
views
Getting an error while trying to train my model in train_function (Empty Logs)
I am trying to train a GPT2 model on the wikipedia text. While doing so I get the following error:
ValueError: Unexpected result of `train_function` (Empty logs). Please use
`Model.compile(..., ...
0
votes
0
answers
73
views
GPT-2: Setting biases as -1 billion
I'm currently trying to predict upcoming words given an input text chunk, but I want to "mask" the last n words of the input text by setting the attention weights to 0 (or something very ...
0
votes
0
answers
26
views
model.bert() through the slicing error can anyone let me know why is it?
with torch.no_grad():
logits = torch.zeros(len(definitions), dtype=torch.double).to(DEVICE)
for i, bert_input in list(enumerate(features)):
logits[i] = model.ranking_linear(
...
0
votes
0
answers
114
views
While trying to generate text using GPT-2 the custom loss function accesses PAD_TOKEN_ID
While training the custom loss function tries to access the PAD_TOKEN_ID resulting in the below error.50257 is the PAD_TOKEN_ID and the vocab size of GPT-2
InvalidArgumentError: {{function_node ...
1
vote
2
answers
171
views
Disable layers in GPT-2 model
I'm currently using a GPT-2 model that was trained on German texts. I would like to generate the next word in a text given a context chunk, but instead of using the whole model to predict the next ...
0
votes
1
answer
128
views
Kaggle Code doesn't download "gpt2" language model
I am using kaggle code to download gpt2 language model.
from transformers import AutoTokenizer, AutoModelForCausalLM
device = "cuda" if torch.cuda.is_available() else "cpu"
...
0
votes
1
answer
102
views
GPT2 special tokens: Ignore word(s) in input text when predicting next word
I just started using GPT2 and I have a question concerning special tokens:
I'd like to predict the next word given a text input, but I want to mask some words in my input chunk using a special token. ...
0
votes
0
answers
354
views
how to fix "KeyError: 0" in the hugging face transformer train() function
hello guys please i am in dying need of your help .
i am trying to fine-tune the gpt2-meduim model with the hugging face transformer and i ran into this error just when i wanted to start the training &...
0
votes
1
answer
404
views
Huggingface Transformers (PyTorch) - Custom training loop doubles speed?
I've found something quite strange when using Huggingface Transformers with a custom training loop in PyTorch.
But first, some context: I'm currently trying to fine tune a pretrained GPT2 small (...
2
votes
1
answer
373
views
How to save the gpt-2-simple model after training?
I trained the gpt-2-simple chat bot model but I am unable to save it. It's important for me to download the trained model from colab because otherwise I have to download the 355M model each time (see ...
1
vote
0
answers
29
views
In which form should be dataset in NLP model?
I try to make fine-tuning of model tinkoff-ai/ruDialoGPT-medium. In which form should be my dataset? The base generation is in form:
@@ПЕРВЫЙ@@ привет @@ВТОРОЙ@@ привет @@ПЕРВЫЙ@@ как дела? @@ВТОРОЙ@@
...
1
vote
0
answers
364
views
Questions about masks of padding in GPT
The GPT series models use the decoder of Transformer, with unidirectional attention. In the source code of GPT in Hugging Face, there is the implementation of masked attention:
self.register_buffer(
...
0
votes
1
answer
176
views
I am getting error here torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) when I call trainer.train() function of GPT2 model
I am new to NLP and I was trying gpt2 to train on my own data.
from transformers import GPT2Config, GPT2LMHeadModel, GPT2Tokenizer, TextDataset, DataCollatorForLanguageModeling, Trainer, ...
0
votes
0
answers
56
views
Getting error "Container localhost does not exist. (Could not find resource: localhost/model/wpe)" while generating with gpt2-simple
I am trying to generate text using the GPT-2 language model using the gpt2-simple library. The training process worked fine, but I am running into an error when I try to generate text using the ...
0
votes
0
answers
50
views
Time and cost to train Distill GPT-2 model on BookCorpus using AWS EC2
I am trying to calculate the time it would take to train a Distill GPT2 model on BookCorpus dataset using multiple EC2 instances for the purpose of language modeling.
What is the method for ...
0
votes
1
answer
620
views
How to fine-tune gpt2 with a custom set of unlabelled document
I'm newbie to GPT2 fine-tuning. My goal is to fine-tune GPT-2 (or BERT) on a my own set of document, in order to be able to query the bot on a topic contained in these documents, and receive an answer....
0
votes
1
answer
151
views
How do I restart Hugging Face Transformer GPT2 finetuning?
I'm trying to restart fine-tuning but it starts from the beginning. Is this normal?
I want to resume fine-tuning using a saved checkpoint. However, when I replace the line model = GPT2LMHeadModel....
0
votes
0
answers
147
views
Finetuning gpt2, validation loss increases with accuracy and f1 score
I am finetuning gpt2 on text classification with the huggingface trainer. I observed that after 2 epochs, my validation loss start to increase, but my validation accuracy and f1 score still increases ...
2
votes
2
answers
501
views
How to deal with stack expects each tensor to be equal size eror while fine tuning GPT-2 model?
I tried to fine tune a model with my personal information. So I can create a chat box where people can learn about me via chat gpt.
However, I got the error of
RuntimeError: stack expects each tensor ...