Questions tagged [seq2seq]
Seq2Seq is a sequence to sequence learning add-on for the python deep learning library.
306
questions
0
votes
0
answers
28
views
How to train a language model for my data
I have a dataset of IDs that are meaningful to me.
I want to use language models to generate IDs based on a few IDs that I give as a starting point.
Let's say my dataset is like a sequence of IDs in ...
0
votes
1
answer
35
views
seq2seq inference outputs wrong results despite high accuracy
I am training a seq2seq model following Keras tutorial https://keras.io/examples/nlp/lstm_seq2seq/, the same code but a different dataset.
Here is the main model code for reference:
Code snippet for ...
0
votes
1
answer
52
views
How Seq2Seq Context Vector is generated?
I have studied the theory of seq2seq model but I couldn't clearly understand that what exactly is context vector and how is it generated. I know it summarizes the meaning of to-be-encoded sequence ...
0
votes
0
answers
21
views
When to update weights in RL model
I am building a chatbot model using Policy gradient Reinforcement Learning. The agent is a Seq2seq LSTM based model. I am using cross entropy loss. Do I need to update the weights of the model after ...
0
votes
0
answers
40
views
Training seq2seq LM over multiple iterations in PyTorch, seems like lack of connection between encoder and decoder
My seq2seq model seems to only learn to produce sequences of popular words like:
"i don't . i don't . i don't . i don't . i don't"
I think that might be due to a lack of actual data flow ...
0
votes
0
answers
39
views
Pytorch LSTM: Encoder/Decoder architecture for different input/output sequence length
I have input data which consists of 9 variables with a sequence length of 92. I want to predict a sequence of 7 other variables, however, this one has a sequence length of 4. Input shapes into my ...
0
votes
1
answer
61
views
loss is drastically decreasing whereas BLEU score stays at zero during training of the seq2seq RNN for machine translation
I'm trying to train an RNN for machine translation, using LSTM. However,the BLEU at the first batch decreases to zero and stay at this level during all the training. At the same time loss is ...
0
votes
0
answers
12
views
How to forecast for a time series where the input and output sequences have different time steps and datasets for both are multivariate?
Case: Lets take minute-wise 7-featured data for 4 hours a day for last 5 years and take P periods (per day) as input to output a sequence of Q periods (per day). I tried using feed forward dense ...
0
votes
0
answers
81
views
How to extract last hidden state from bidirectional LSTM (encoder decoder structure)?
For my uni assignment, I have given a notebook of a seq2seq model using PyTorch. Some of the code within the methods has been removed and I have to fill it in. There are some To-Do's in the code that ...
0
votes
0
answers
18
views
Custom tokenization rules
Is it possible to configure custom tokenization rules for a field that will break words containing letters and numbers into separate tokens? For example, I'd like the string "S1E1e2" to be ...
1
vote
0
answers
43
views
ROUGE-1.5.1.pl problem when train seq2seq model
It is a problem about seq2seq-summarizer. ( more details see)I run these codes on Google colab. I just train it one epoch,and it returns an error.
the imformation about the error
and I download ROUGE-...
0
votes
0
answers
26
views
Validation loss never decreases during training
enter image description hereI encountered one perplexing situation: I trained a Transformer model for conditional music generation, finding the training loss decrease while the validation loss keeps ...
0
votes
0
answers
41
views
I have a error when i'm working with seq2seq model: Module 'tensorflow' has no attribute 'contrib'
When i'm having create a tf.data dataset
BUFFER_SIZE = len(input_tensor_train)
BATCH_SIZE = 64
embedding_dim = 256
units = 1024
vocab_inp_size = len(inp_lang.word2idx)
vocab_tar_size = len(targ_lang....
0
votes
0
answers
18
views
Key Error on Tensorflow in Style transfer problem
def evaluate(encoder, decoder, in_lang, max_length=MAX_LENGTH):
if use_cuda:
in_lang = in_lang.cuda()
input_variable = Variable(in_lang)
input_variable = input_variable.unsqueeze(0)...
0
votes
0
answers
61
views
"InvalidArgumentError: Graph execution error:" for Seq2seq model with encoder having same input and output pairs
I'm trying to train a seq2seq model, but the encoder of the model has to be trained with same input and output pairs. Basically I am trying to only train the encoder with the two pairs and freeze it, ...
0
votes
2
answers
228
views
Why did the Seq2SeqTrainer not stop when the EarlyStoppingCallback criteria is met?
When trying to use EarlyStopping for Seq2SeqTrainer, e.g. patience was set to 1 and threshold 1.0:
training_args = Seq2SeqTrainingArguments(
output_dir='./',
num_train_epochs=3,
...
0
votes
1
answer
108
views
LSTM seq2seq model in R does not seem to use trained model for predictions
For a project i'm trying to create a function that can "translate" names written in latin characters into IPA (phonetic).
I've found an example of a sequence2sequence model in TensorFlow for ...
1
vote
0
answers
27
views
What is “Ensemble of 5 reversed LSTMs” in seq2seq
While reading the seq2seq paper (Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.), the ...
0
votes
0
answers
29
views
Bad performance in seq2seq model for human motion forecasting
I am working on a human motion prediction project, where given an input sequence, n_outputs samples have to be predicted. Each sequence is composed by 31 joints of 3-D coordinates, which translates ...
0
votes
0
answers
30
views
How to use a text dataset that is al ready tokenized at character level?
I have a text dataset that contains already tokenized text, at character level with characters separated by space, space replaced by _ and unknown characters replaced by a #. The goal is to use this ...
1
vote
1
answer
48
views
Keras seq2seq model Output Shapes
I am working on keras seq2seq example here:https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html
What I have understood from the text is in decoder model each ...
0
votes
1
answer
156
views
sequence to vector LSTM model
I have 10 sequence of numerical data which is to be used as input to LSTM.
And these sequences are mapped to a vector with 2 values.
sequences are of equal length.
Please note. Based on the sequence ...
5
votes
1
answer
135
views
Equivalent of tf.contrib.legacy_seq2seq.attention_decoder in tensorflow 2 after upgrade
I have the following code in TensorFlow 1.0. I tried to migrate it to TensorFlow 2.0 using tf_upgrade_v2 script. However, it didnt find an equivalent function in the tf-2 compact version.
I was ...
2
votes
0
answers
97
views
Seq2Seq(GNN+RNN) - Odd predictions despite optimized loss
I’d like to ask for your advise/expertise on an issue that I am currently facing.
Summary:
I am training a Seq2Seq model that generates a natural language question based on a graph. Train and ...
-1
votes
1
answer
205
views
Tensorflow ValueError: No gradients provided for any variable:
I was trying to do a custom loss function but I get the error.
This is supposed to be cyclegan for text
This is the loss function:
def sloss(ytrue, ypred):
nump = 0
print(nump)
for i in range(...
0
votes
0
answers
32
views
Error getting gradients from model converted from tf1 to tf2 with tfa.seq2seq layers: "'NoneType' object has no attribute 'outer_context'"
After converting a model from tf1 to tf2, I cannot use the tf2 model with tf.GradientTape().
Some useful information:
The model uses tfa.seq2seq layers, which are most likely the source of the error (...
2
votes
0
answers
23
views
What ML model to predict a sequence of numbers from another sequence
I am trying to predict a time series from another time series. The input and the output have the same length but they have a different structure (The input is more noisy), the output has a nice ...
0
votes
0
answers
36
views
how do i add beam search in inference function tensorflow model
I'm having a hard time adding the beam search to this function.
the initial search was always taking the max probablity in each position ( greedy search), now that i'm trying to add a loop to generate ...
0
votes
1
answer
40
views
How can I use legacy_seq2seq.embedding_rnn_seq2seq in tensorflow?
The problem is i am unable to find seq2seq in new tensorflow library
Here's the code-
decoderOutputs, states = tf.legacy_seq2seq.embedding_rnn_seq2seq(
self.encoderInputs, # List<[batch=?, ...
0
votes
0
answers
22
views
The state of decoder of Seq2Seq ML model struck to a single most probable output despite changing inputs
Convergence issues with my attention-based Seq2Seq Model.
I'm trying to implement Bahdanau Attention for my Sequence to Sequence Machine Translation model for over a month without any success and I ...
0
votes
0
answers
304
views
Bert Model Seq2Seq Hugginface translation task
I am trying to fine-tune a Bert2Bert Model for the translation task, using deepspeed and accelerate.
I am following the suggested post and the examples/pytorch/translation both by Hugginface.
...
0
votes
0
answers
63
views
Keras Bidirectional LSTM seq2seq inference model expects 3 inputs but only receives 1, even though I am passing in 3 inputs
I am creating a language model with a bidirecitonal LSTM, seq2seq model.
I have created the model and trained it successfully:
lstm_units = 100
# Set up embedding layer using pretrained weights
...
0
votes
1
answer
142
views
Understanding states of a bidirectional LSTM in a seq2seq model (tf keras)
I am creating a language model: A seq2seq model with 2 Bidirectional LSTM layers. I have got the model to train and the accuracy seems good, but whilst stuck on figuring out the inference model, I've ...
0
votes
0
answers
38
views
How to handle repetition problem in seq2seq text summarization problem
For Seq2seq text summarization i am facing issue related to repetition. What are the best way to handle the repetition problem in seq2seq text summarization:
at the time of decoding i am getting ...
1
vote
1
answer
1k
views
TypeError: from_pretrained() got an unexpected keyword argument 'file_name'
I'm trying to quantize a seq2seq model (M2M100) using optimum library provided by Huggingface. As per this guide, I'm trying to quantize the encoder and decoder one by one but that requires me to ...
0
votes
0
answers
58
views
ValueError: Dimensions must be equal, but are 1024 and 256 for
I am trying to use Keras seq2seq model but kept having an error.
ValueError: Dimensions must be equal, but are 1024 and 256 for '{{node mul}}
= Mul[T=DT_FLOAT](Sigmoid_1, init_c)' with input shapes: [?...
0
votes
0
answers
13
views
Does pattern in sentence edits affect the performance of sentence correction seq2seq model
I am trying to train a seq2seq model using T5 transformer for sentence correction task. I am using StackOverflow dataset for the training and evaluation process. The dataset contains original and ...
2
votes
0
answers
317
views
What's the correct way of inference for a transformer model?
I'm beginner learning to build a standard transformer model based on PyTorch to solve an univariate sequence-to-sequence regression problem. The codes are written referring to the tutorial of PyTorch, ...
0
votes
0
answers
121
views
Building Keras Seq2Seq Inference Model
I am building a Seq2Seq Model with the Encoder-Decoder Architecture. The model is aimed to summarise input text. The training model has been built and the training seems fine. Below is the source code ...
0
votes
0
answers
75
views
Bidirectional LSTM Seq2Seq with Residual connections in decoder
I am trying to build a seq2seq model including bidirectional LSTMs in both encoder and decoder parts. I also want to add residual connections to the decoder part, i.e., to add the input of each ...
0
votes
0
answers
215
views
Output directory is empty in Trainer
With my script the model is correctly trained and the results are printed, but the results directory is empty. How is that? What is lacking? I think I should have the files described in this answer.
...
0
votes
0
answers
342
views
You have to specify either decoder_input_ids or decoder_inputs_embeds
I want to fine-tune Facebook's M2M100 model, but I get the following error: You have to specify either decoder_input_ids or decoder_inputs_embeds.
The thing is, in batch.keys() I only get dict_keys(['...
0
votes
0
answers
23
views
Character-level seq2seq model for translation and beam search
I was trying to implement seq2seq translation model at character level along with beam search by referring the tensorflow documentation.
https://www.tensorflow.org/addons/tutorials/...
0
votes
0
answers
37
views
NLP with PyTorch - RuntimeError: shape '[32, 128, 1]' is invalid for input of size 61440
I'm trying to run this code for the attention model in NLP.
class DecoderAttn(nn.Module):
def __init__(self, output_dim, emb_dim, hid_dim, n_layers, attn_dim):
super().__init__()
self.hid_dim =...
0
votes
1
answer
775
views
How to add keras attention layer in seq2seq encoder decoder model?
I was trying to perform character level translation using keras seq2seq model, but I'm unable to add attention layer.
I took the reference of keras seq2seq documentation.
https://keras.io/examples/nlp/...
0
votes
1
answer
130
views
What does keras 'accuracy' mean in seq2seq model?
I'm trying to build a seq2seq model to predict sequence. The most basic model was built, but I'm having trouble with understanding what 'metric=['accuracy']' means here.
Below is the link that is very ...
0
votes
0
answers
63
views
How to translate my own sentence using Attention mechanism?
For every language translation repos, I can see the existing sentences which are translated. If i give my own sentence ( evnthough the words of my own sentence are existed in the dataset) the ...
0
votes
0
answers
89
views
Use 1 tokenizer or 2 tokenizers for translation task?
I’ve seen several tutorials about seq2seq tasks like translation. They usually use 2 tokenizers trained on corpus, one for source language and the other for target language. However, in huggingface’s ...
0
votes
0
answers
116
views
How to implement a transformer that takes a sequence of float arrays and outputs a sequence of float array
I have a time-series problem with 966 samples, 139 time-steps, and fixed length float vectors of tunable size n. As output, I need a transformer model that takes a sequence (time-steps) of fixed ...
0
votes
1
answer
394
views
How does the finetune on transformer (t5) work?
I am using pytorch lightning to finetune t5 transformer on a specific task. However, I was not able to understand how the finetuning works. I always see this code :
tokenizer = AutoTokenizer....