Questions tagged [gpt-2]

Use this tag with Generative Pre-trained Transformer 2 (GPT-2). Do not use with GPT-3 or the ad tagging library (GPT).

Filter by
Sorted by
Tagged with
0 votes
1 answer
23 views

How use past_key_values in pytorch for caching

why using past_key_values of "hello, my dog is" plus the input id of "cute" doesn't output the same as using "hello, my dog is cute" In my understanding the ...
juan manuel kersul's user avatar
0 votes
0 answers
49 views

The maximum value of input_ids (50257) must be smaller than the embedding layer's input dimension (50257)

I am trying to create a categorical model that produces one of the three result for some data input: True, False, Invalid. as mentioned in the title I get this error when I try to evaluate the model, ...
Ghazi's user avatar
  • 613
0 votes
1 answer
35 views

Why can we set LLM's input and output to be the same when fine tuning on text generation task?

I'm trying to fine tune my GPT-2 model for song lyrics text generation, and I have a couple of song lyrics on hand. However, I'm confused about how to fine tune GPT-2 model that doesn't have standard ...
orange0629's user avatar
3 votes
0 answers
18 views

compute LM conditional probability of a text given a prefix

I am seeking a pythonic way to compute the probability of text chunk condition on a text prefix. Example below: from transformers import GPT2Tokenizer, GPT2LMHeadModel tokenizer = GPT2Tokenizer....
Garvey's user avatar
  • 1,249
1 vote
0 answers
14 views

Perplexity metric for GPT2 model is lower for non-English text

I am currently working on a project of calculating perplexities of various causal LLMs for different languages to estimate their behaviour if there is an input in a form of the language, that ...
Nikita Volkov's user avatar
0 votes
0 answers
69 views

GPT2 and HuggingFace Transformers: repeating user_input in response

I am just getting into making my first model using GPT 2 and the HuggingFace Transformers library in Python as my gateway into understanding model training. I'm running this on Python 3.7 with the ...
Yuuty's user avatar
  • 123
0 votes
0 answers
24 views

Softmax output and probabilities not matching up?

I'm trying to test how well a GPT model can classify verbs according to the left-side context in a given input sentence with a masked term. For example, Input sentence: "The ballerinas' costumes ...
karak87rt0's user avatar
0 votes
1 answer
39 views

Why is my streamlit app not correctly summarizing my mp3-transcription?

I'm working on a Streamlit app that processes MP3 files. The main steps include: Uploading an MP3 file. Splitting the audio into smaller chunks using pydub. Transcribing these chunks using OpenAI. ...
Patrick Schmidt's user avatar
0 votes
1 answer
117 views

How do fix GPT2 Tokenizer error in Langchain map_reduce (LLama2)?

I'm using AWS Sagemaker Jumpstart model for Llama2 13b: meta-textgeneration-llama-2-13b-f On running a Langchain summarize chain with chain_type="map_reduce" I get the below error. I do not ...
apprunner2186's user avatar
0 votes
0 answers
19 views

ValueError: Expected input batch_size (1052) to match target batch_size (508) when fine tuning GPT 2 model

Hello there I'm attempting to train a GPT 2 model how to summarize passages without compromising their emotional impact. Consider summarizing a chapter from a book, but we want the reader to ...
Damika's user avatar
  • 652
1 vote
0 answers
191 views

Invalid key: 409862 is out of bounds for size 0

How I can fix this: I writed code for training GPT-2 on dataset by Hugging Face, but I have an error and don't know why I got this error: ---------------------------------------------------------------...
Vovancho's user avatar
0 votes
0 answers
34 views

'utf-8' codec can't decode byte 0xc3 error when using tensorflow.keras.layers import TextVectorization

I am trying to execute the steps given in a blog post (https://stackabuse.com/gpt-style-text-generation-in-python-with-tensorflowkeras/) but getting the error in the below block: vectorize_layer.adapt(...
Vaibhav's user avatar
  • 102
0 votes
0 answers
66 views

GPT-2 PyTorch Custom Training

I have this implementation of OpenAI GPT-2 LLM in PyTorch. Can you please help me write a training loop for it, if my dataset looks like this text file: <|user|>How are you?<|bot|>I’m fine&...
Yuki Arimo's user avatar
0 votes
0 answers
18 views

tf.compat.v1.estimator.Estimator(): NameError: name 'model_fn' is not defined.Getting errors in add_argument as well. Not recognizing paths mentioned

I am trying to create a pet LLM using GPT-2 following instructions here: https://thomascherickal.medium.com/how-to-create-your-own-llm-model-2598615a039a The code gives syntax error while calling tf....
Mohit Agrawal's user avatar
0 votes
0 answers
54 views

Title: Generating Sentences with TRL while Maintaining Sentiment - Issue with "AutoModelForCausalLMWithValueHead"

I am currently working on generating sentences with TRL (Transformers Reinforcement Learning) while preserving the same sentiment as the sample sentences. However, I've come across an issue with the ...
user11849691's user avatar
0 votes
1 answer
203 views

Hugging Face Inference API returning short generated text with GPT-2 model

I'm using the Hugging Face API with a GPT-2 model to generate text based on a prompt. However, I'm encountering an issue where the generated text is consistently too short, even though I'm specifying ...
ma.ca's user avatar
  • 1
0 votes
1 answer
569 views

How to generate text using GPT2 model with Huggingface transformers?

I wanted to use GPT2Tokenizer, AutoModelForCausalLM for generating (rewriting) sample text. I have tried transformers==4.10.0, transformers==4.30.2 and --upgrade git+https://github.com/huggingface/...
user11849691's user avatar
1 vote
0 answers
69 views

GPT2 LLM fine-tuned model not generating expected answer

I am finetuning gpt2 model to answer questions with given faq.json. There is some issue with the answer generated by below code. I am assuming I have not done encoding/decoding of questions and ...
tagg's user avatar
  • 383
0 votes
0 answers
44 views

Non-meaningful response from finetuned GPT-2 model

I am experimenting with the abilities of GPT-2 for question answering aiming at making a good task-based chatbot. I trained my model on the air_dialogue dataset from huggingface https://huggingface....
Chukwujike's user avatar
0 votes
0 answers
22 views

FineTune GPT2 on Insurance Domain data

I am new to LLM and I am trying to finetune GPT2 from Huggingface on Insurance domain data. I am not getting results from the Trained data instead I am getting different results. My Training Data is a ...
vjsat2k18's user avatar
2 votes
1 answer
52 views

How to refine a trained model in gpt2?

Im currently trying to work on text generation with my own text. I have trained my model with gpt2 with my own text. But it is giving random answers. For some questions it is giving me relevant ...
Bhavani Priya's user avatar
0 votes
0 answers
147 views

How to train gpt2 model to learn from the training text I have given?

I'm trying to train and fine tune my gpt2 model with my own sample training document. I'm using the code similar to this: https://www.kaggle.com/code/changyeop/how-to-fine-tune-gpt-2-for-beginners . ...
Bhavani Priya's user avatar
2 votes
1 answer
4k views

What is the cause of HFValidationError in this code and how do I resolve this error?

My python code in Chaquopy android studio Project: import torch as tc from transformers import GPT2Tokenizer, GPT2Model def generate_text(txt): """ Generate chat https://...
baponkar's user avatar
  • 270
0 votes
1 answer
128 views

Unsupervised fine-tuning on custom documents after the supervised fine tuning on general question-answers dataset. Will it be useful for GPT-2 model?

I know the formal way of training a GPT2 model on custom documents is to first do semi-supervised fine tuning on the text of the documents followed by supervised fine-tuning on question answers from ...
Ratna Sambhav's user avatar
0 votes
0 answers
60 views

tf.compat.v1.estimator.Estimator(): NameError: name 'model_fn' is not defined

I am trying to create a pet LLM using GPT-2 following instructions here: https://thomascherickal.medium.com/how-to-create-your-own-llm-model-2598615a039a The code gives syntax error while calling tf....
sm535's user avatar
  • 587
0 votes
0 answers
47 views

TypeError: argmax(): argument 'input' (position 1) must be Tensor, not numpy.ndarray

I am traning a model GPT-2 using my curated dataset, and getting the following error. When I am trying to debug any issue, a new error comes. Can anyone helpme to fix my script so that it can run. The ...
Sushant Pandey's user avatar
6 votes
2 answers
962 views

Generate the probabilities of all the next possible word for a given text

i have the following code import transformers from transformers import pipeline # Load the language model pipeline model = pipeline("text-generation", model="gpt2") # Input ...
datadigger's user avatar
1 vote
0 answers
67 views

Expected input batch_size (28) to match target batch_size (456), Changing batch size increase the target batch size with GPT2 model

I was practising fine-tuning a gpt2 model on a simple question-answer dataset when I encountered this error. I have studied other answers, but my input dataset shapes look fine. def tokenize_data(...
Irfan Yaqub's user avatar
0 votes
1 answer
229 views

IndexError: index out of range in self while using GPT2LMHeadModel.from_pretrained("gpt2")

I am working on this question answering code and using pretrained GPT2LMHeadModel. But after tokenization when I pass the inputs and attention mask to the model it is giving index error. My code: ...
Md Tahmid Hasan Fuad's user avatar
0 votes
0 answers
307 views

TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]] when training GPT-2

I'm learning how to train generative models using the transformers library from HuggingFace, however, I keep having this error: TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[...
Erika's user avatar
  • 151
0 votes
1 answer
172 views

My gpt2 code generates a few correct words and then goes into a loop of generating the same sequence again and again

The following gpt2 code for sentence completion generates a few good sentences and then ends in a loop of repetitive sentences. from transformers import GPT2LMHeadModel, GPT2Tokenizer ...
steve landiss's user avatar
0 votes
0 answers
171 views

Getting an error while trying to train my model in train_function (Empty Logs)

I am trying to train a GPT2 model on the wikipedia text. While doing so I get the following error: ValueError: Unexpected result of `train_function` (Empty logs). Please use `Model.compile(..., ...
steve landiss's user avatar
0 votes
0 answers
73 views

GPT-2: Setting biases as -1 billion

I'm currently trying to predict upcoming words given an input text chunk, but I want to "mask" the last n words of the input text by setting the attention weights to 0 (or something very ...
Merle's user avatar
  • 125
0 votes
0 answers
26 views

model.bert() through the slicing error can anyone let me know why is it?

with torch.no_grad(): logits = torch.zeros(len(definitions), dtype=torch.double).to(DEVICE) for i, bert_input in list(enumerate(features)): logits[i] = model.ranking_linear( ...
Faizan Munsaf's user avatar
0 votes
0 answers
114 views

While trying to generate text using GPT-2 the custom loss function accesses PAD_TOKEN_ID

While training the custom loss function tries to access the PAD_TOKEN_ID resulting in the below error.50257 is the PAD_TOKEN_ID and the vocab size of GPT-2 InvalidArgumentError: {{function_node ...
S_2010's user avatar
  • 1
1 vote
2 answers
171 views

Disable layers in GPT-2 model

I'm currently using a GPT-2 model that was trained on German texts. I would like to generate the next word in a text given a context chunk, but instead of using the whole model to predict the next ...
Merle's user avatar
  • 125
0 votes
1 answer
128 views

Kaggle Code doesn't download "gpt2" language model

I am using kaggle code to download gpt2 language model. from transformers import AutoTokenizer, AutoModelForCausalLM device = "cuda" if torch.cuda.is_available() else "cpu" ...
agongji's user avatar
  • 127
0 votes
1 answer
102 views

GPT2 special tokens: Ignore word(s) in input text when predicting next word

I just started using GPT2 and I have a question concerning special tokens: I'd like to predict the next word given a text input, but I want to mask some words in my input chunk using a special token. ...
Merle's user avatar
  • 125
0 votes
0 answers
354 views

how to fix "KeyError: 0" in the hugging face transformer train() function

hello guys please i am in dying need of your help . i am trying to fine-tune the gpt2-meduim model with the hugging face transformer and i ran into this error just when i wanted to start the training &...
nkdtech's user avatar
  • 33
0 votes
1 answer
404 views

Huggingface Transformers (PyTorch) - Custom training loop doubles speed?

I've found something quite strange when using Huggingface Transformers with a custom training loop in PyTorch. But first, some context: I'm currently trying to fine tune a pretrained GPT2 small (...
Twisted Tea's user avatar
2 votes
1 answer
373 views

How to save the gpt-2-simple model after training?

I trained the gpt-2-simple chat bot model but I am unable to save it. It's important for me to download the trained model from colab because otherwise I have to download the 355M model each time (see ...
argo's user avatar
  • 21
1 vote
0 answers
29 views

In which form should be dataset in NLP model?

I try to make fine-tuning of model tinkoff-ai/ruDialoGPT-medium. In which form should be my dataset? The base generation is in form: @@ПЕРВЫЙ@@ привет @@ВТОРОЙ@@ привет @@ПЕРВЫЙ@@ как дела? @@ВТОРОЙ@@ ...
Ubuty_programmist_7's user avatar
1 vote
0 answers
364 views

Questions about masks of padding in GPT

The GPT series models use the decoder of Transformer, with unidirectional attention. In the source code of GPT in Hugging Face, there is the implementation of masked attention: self.register_buffer( ...
LocustNymph's user avatar
0 votes
1 answer
176 views

I am getting error here torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) when I call trainer.train() function of GPT2 model

I am new to NLP and I was trying gpt2 to train on my own data. from transformers import GPT2Config, GPT2LMHeadModel, GPT2Tokenizer, TextDataset, DataCollatorForLanguageModeling, Trainer, ...
Shubhankar Kumar's user avatar
0 votes
0 answers
56 views

Getting error "Container localhost does not exist. (Could not find resource: localhost/model/wpe)" while generating with gpt2-simple

I am trying to generate text using the GPT-2 language model using the gpt2-simple library. The training process worked fine, but I am running into an error when I try to generate text using the ...
Kit's user avatar
  • 1
0 votes
0 answers
50 views

Time and cost to train Distill GPT-2 model on BookCorpus using AWS EC2

I am trying to calculate the time it would take to train a Distill GPT2 model on BookCorpus dataset using multiple EC2 instances for the purpose of language modeling. What is the method for ...
Troi's user avatar
  • 43
0 votes
1 answer
620 views

How to fine-tune gpt2 with a custom set of unlabelled document

I'm newbie to GPT2 fine-tuning. My goal is to fine-tune GPT-2 (or BERT) on a my own set of document, in order to be able to query the bot on a topic contained in these documents, and receive an answer....
solecuore's user avatar
0 votes
1 answer
151 views

How do I restart Hugging Face Transformer GPT2 finetuning?

I'm trying to restart fine-tuning but it starts from the beginning. Is this normal? I want to resume fine-tuning using a saved checkpoint. However, when I replace the line model = GPT2LMHeadModel....
Blank256's user avatar
0 votes
0 answers
147 views

Finetuning gpt2, validation loss increases with accuracy and f1 score

I am finetuning gpt2 on text classification with the huggingface trainer. I observed that after 2 epochs, my validation loss start to increase, but my validation accuracy and f1 score still increases ...
HCY's user avatar
  • 1
2 votes
2 answers
501 views

How to deal with stack expects each tensor to be equal size eror while fine tuning GPT-2 model?

I tried to fine tune a model with my personal information. So I can create a chat box where people can learn about me via chat gpt. However, I got the error of RuntimeError: stack expects each tensor ...
Chau Loi's user avatar
  • 1,076

1
2 3 4 5