Questions tagged [word-embedding]
For questions about word embedding, a language modelling technique in natural language processing. Questions can concern particular methods, such as Word2Vec, GloVe, FastText, etc, or word embeddings and their use in machine learning libraries in general.
1,039
questions
0
votes
0
answers
20
views
Stuck with SVM classifier using word embeddings/torchtext in NLP task
I'm currently on an task where I need to use word_embedding feature, glove file and torchtext with SVM classifier. I have created a sperate function for it where this is what the implementation of ...
0
votes
2
answers
31
views
How does gensim calculate sentence embeddings when using a pretrained fasttext model?
According to this answer, sentence similarity for FastText is calculated with one of two ways (depending if the embeddings are created superviser or unsupervised)
The mean of the normalized word ...
0
votes
1
answer
10
views
Evaluate FastText embeddings
I want to evaluate my FastText model (trained on my own corpus).
For semantic meaning I understand that we can use a dataset containing several pairs of two words which have been scored by humans, and ...
-1
votes
0
answers
11
views
Predicting word embedding directly from softmax layer without argmax using machine learning model
I want to build a model such that which will predict the corresponding word2vec embedding directly from the softmax layer without taking argmax . How will be the model architecture looks like and what ...
1
vote
0
answers
15
views
Understanding Signal Representation and Processing using Continuous Bag of Words (CBOW) Model
I'm attempting to improve the representation of my signal at each time step using the CBOW method. However, my signal consists of 1000 time steps, which is too large to work with. To address this, I ...
0
votes
2
answers
22
views
Topic modeling with word embeddings
I am currently trying to create an LDA model with word embeddings. Here is the code:
from gensim.models.ldamodel import LdaModel
from gensim.corpora.dictionary import Dictionary
from gensim.test.utils ...
2
votes
1
answer
126
views
Semantic searching using Google flan-t5
I'm trying to use google flan t5-large to create embeddings for a simple semantic search engine. However, the generated embeddings cosine similarity with my query is very off. Is there something I'm ...
0
votes
0
answers
49
views
OutOfMemoryError when I create model embeddings
Just started learning huggingface transformers. I am trying to create embeddings of a large amount of text but I always run into outOfMemoryErrors. I am not sure what I am doing wrong. I am new to ...
0
votes
0
answers
28
views
running locale text embedding projector with Pytorch
I want to run a text embedding projector locally using Pytorch (not tensorflow).
Everything is installed and running this comment bring up the TensolBoard empty:
import keyword
import torch
meta = []
...
0
votes
0
answers
15
views
How to get vocabulary from WordEmbeddingsModel in sparknlp
I need to create an embedding matrix from embeddings generated by WordEmbeddingsModel in sparknlp. Until now i have this code :
from sparknlp.annotator import *
from sparknlp.common import *
from ...
1
vote
1
answer
62
views
What does the embedding elements stand for in huggingFace bert model?
Prior to passing my tokens through encoder in BERT model, I would like to perform some processing on their embeddings. I extracted the embedding weight using:
from transformers import TFBertModel
# ...
1
vote
0
answers
15
views
How to find Sentence Transformer support languages?
I want to get the sentence embedding results to find the sentence similarities in my NLP project. Since I am working with a low-resource language (Sinhala), I want to know whether any ...
1
vote
1
answer
41
views
Embedding version control of the weaviate
In Weaviate, the vector engine, I wonder how this can handle version issue of embedding model.
For instance, considering the (trained) word2vec model, embedded vectors from different models must be ...
0
votes
0
answers
18
views
Text classification using NER, SRL, etc
I'm working on a task of classifying text complaints and I extracted some features like Named Entities, Events, Time Expressions, Semantic Role Labels, etc. I want to classify the text according to ...
0
votes
0
answers
52
views
Julia/Flux creating a model correctly - using Chain Embedding layer reshaping & Dense layers
I am trying to follow an old article about word embedding in Julia language with Flux: https://spcman.github.io/getting-to-know-julia/deep-learning/nlp/flux-embeddings-tutorial-1/
As it is an old ...
0
votes
0
answers
13
views
Running SVM on positional embeddings using keras for text classification
How can I run SVM on a large text classification dataset for detecting fake news of 400 thousand entries that uses positional encoding for embeddings from keras and has a maximum sentence length of 15 ...
0
votes
1
answer
27
views
Can not run the code of this repository - NETL-Automatic-Topic-Labelling-
I am trying to run this code - Automatic Labelling of Topics with Neural Embeddings
The problem is that they did not mention what versions they used for the libraries and tools they used. Sadly, not ...
0
votes
1
answer
25
views
How to encode the shortest dependency path between words in sentence in neural network?
I am doing a first project on relation extraction between two entities in sentences in NLP. I use an LSTM model, and in addition to inputting the word embedding, I want to input the shortest ...
0
votes
0
answers
29
views
How to use gpu when computing word embeddings in textEmbed()?
I noticed that the textEmbed function has a device option to choose between cpu or gpu.
But when running textEmbed with "device" set to "gpu" I get the message: "Unable to use ...
0
votes
0
answers
36
views
The most performant and proper way to get similarities between large embeddings/vectors
I have 2 data frames with 69 & ~230.000 rows and (KEY, Embedding) columns.
The embedding columns are array type and each has a length of 768 which I obtained from a fine-tuned transformer model (...
0
votes
0
answers
14
views
any word embedding model for medical contexts?
Hey so I'm searching for a word embedding model trained on medical data to write a program that finds a medical diagnosis from free text.
I've found a model called bioSentVec but I'm not sure that ...
0
votes
0
answers
45
views
Including a covariate in a word embedding model in R using text2vec and quanteda packages
I am trying to build a word embedding model in r with the following code:
library(quanteda)
library(text2vec)
fcm_ <- fcm(tokens, context = "window", count = "weighted", ...
0
votes
0
answers
14
views
Select Topic Words from Clusters
I am following this solution for clustering: https://towardsdatascience.com/clustering-contextual-embeddings-for-topic-model-1fb15c45b1bd
For step four "Select Topic Words from Clusters", I ...
0
votes
0
answers
38
views
SpaCy "en_core_web_trf" gives empty result
To use BERT for word embedding, why does SpaCy "en_core_web_trf" give an empty result?
0
votes
0
answers
20
views
Tensorflow: Using Word Embeddings to Tranform Target Train into Embedding Vectors, does it make sense?
I have been studying TF for only a few months now, so bear with me, and thought of this as a hypothetical question.
For example, I have as input, audio spectrograms of words, and I am training a model ...
0
votes
1
answer
42
views
Gensim Word2Vec produces different most_similar results through final epoch than end of training
I'm using gensim's Word2Vec for a recommendation-like task with part of my evaluation being the use of callbacks and the most_similar() method. However, I am noticing a huge disparity between the ...
0
votes
0
answers
34
views
Text Classification roBERTA word embedding to CNN, accuracy not improving
I am learning CNN and roBERTa word embedding, I create a sentiment analysis with 3 label, -1 for negative, 0 for neutral, 1 for positive. I already have word embedding from roBERTa but when processing ...
1
vote
1
answer
44
views
Python / Word2Vec: How to project a word on two axis e.g. 'man-woman' and 'rich-poor'
How can I project a word W on an X-axis which represents the scale 'man-woman' and on Y-axis which represents the scale 'poor-rich'?
Let's say my word is:
word = model.wv['king']
and I would like to ...
0
votes
1
answer
68
views
'Word2Vec' object has no attribute 'infer_vector'
This is the version of gensim I am using:
Name: gensim
Version: 4.3.0
Summary: Python framework for fast Vector Space Modelling
Home-page: http://radimrehurek.com/gensim
Author: Radim Rehurek
Author-...
1
vote
1
answer
104
views
Calculate cosine similarity between 2 words using BERT
I am trying to calculate the cosine similarity beteween two given words using BERT, but I am getting an error which says:
IndexError: Dimension out of range (expected to be in range of [-1, 0], but ...
0
votes
0
answers
22
views
Why the ouput of nn.Embeddings(vocab_size, dim) chnages on re-running the code for same input string?
I am trying to understand how word embeddings are generated, I've been reading that 1-hot encoded vector is used and it servers as a lookup table but, I want to print that and see, how can I do that. ...
0
votes
0
answers
83
views
The range of similarity in openAI text-embeddings is not [0, 1]
I am using openAI Ada to create text-embeddings. When i calculate the similarity between these embeddings my answer is in the range [0.7, 0.9] approximately. In the rest of my project I would need ...
0
votes
0
answers
23
views
Problem with using multiple inputs with embedding layers in a Keras Model
I am trying to build a basic NN using Keras using multiple input layers, some of which are connected to Embedding layers. I am using the data for Kaggle's Housing Prices competition.
Here is the ...
0
votes
1
answer
183
views
How do I introduce a new subject without extra training in Stable Diffusion?
Suppose I have a dataset of 1000 pokemon, I have 10 images of Pikachu, 10 images of Bulbasaur, etc.
I also have a metadata specifying the name of each pokemon exactly. So from the metadata, I can know ...
0
votes
0
answers
83
views
Is normalization of word embeddings important?
I am doing actor-critic reinforcement learning for an environment that is best represented as a "bag-of-words". For this reason, I have opted to use a single body, multi-head approach for ...
0
votes
1
answer
46
views
Use three transformations (average, max, min) of pretrained embeddings to a single output layer in Pytorch
I have developed a trivial Feed Forward neural network with Pytorch.
The neural network uses GloVe pre-trained embeddings in a freezed nn.Embeddings layer.
Next, the embedding layer splits into three ...
0
votes
1
answer
51
views
Changing shapes of PyTorch tensors and numpy arrays
I'm using CLIP model from huggingface to generate image embeddings, and I'm struggling with the output's shape.
I'm trying to get a numpy array of shape (n, 512) - given n samples and 512 is the ...
0
votes
0
answers
78
views
How to train string feature as word embedding vectors as part of a Linear Regression Tensorflow model?
How do I train string feature as word embedding vectors as part of a linear regression Tensorflow model? I found example online about how to uses Tensorflow to vectorize strings and use these vectors ...
0
votes
0
answers
56
views
While installing BioSentVec I am getting an error for the ```make``` command: The system cannot find the file specified
I am trying to install BioSentVec but getting an error while running the make command. I am following the instructions in the answers to this question. I will focus on the second answer, which is as ...
0
votes
1
answer
75
views
How to work with Doc2Vec and which approach is better training the model on my dataset or using a pretrained model?
I am building a classification model for a dataset of items. Basically, I have 2 columns ex:
Item name
category
unsalted butter
dairy and eggs
cheese
dry grocery
peanut butter cream
dry grocery
I ...
-1
votes
1
answer
67
views
Subword vector in fastText?
I can't figure out what a subword input vector is. I read in the newspaper that the subword is hashed, the subword is the hash code, hash code is a number, not a vector
Ex: Input vector of word eating ...
0
votes
0
answers
42
views
embeddings distribution wrong
I'm having the code below which is supposed to plot word embeddings.
Since it creates a list of embeddings of identical words I would have expected a cluster of points - all embeddings at one point.
...
3
votes
2
answers
379
views
Classic king - man + woman = queen example with pretrained word-embedding and word2vec package in R
I am really desperate, I just cannot reproduce the allegedly classic example of king - man + woman = queen with the word2vec package in R and any (!) pre-trained embedding model (as a bin file).
I ...
1
vote
1
answer
36
views
Generate and Read Word Embeddings with KeyedVectors
I trained a neural network with tensorflow and extracted the weights from the embedding layer to make an array of embeddings. I generated it as a txt file and I can't read it with KeyedVectors
...
0
votes
0
answers
102
views
combining dense (word embedding) and sparse vector (one hot encoding) but my kernel keeps crushing
The function combine_sparse_and_dense_features(dense_vectors, sparse_features) is to combining dense vectors (word embedding) and sparse features (one hot encoding).
Dense vector array shape : (203621,...
0
votes
0
answers
16
views
Where do w in Word2Vect come from?
can someone help me, please. Based on the image, I got a question where's the w1 and w2 actually comes from? I try to break the model down from SoftMax to predict, but I don't know how to find the w1 ...
0
votes
0
answers
74
views
How to get vector embedding of (lat, long) points or GPS trajectories?
I have a data that consists of approx. 1.5M taxi trips. Each of these trips have it's GPS trajectory traveled along as a feature. I would like to embed these trajectories, which are lists of (lat, ...
0
votes
0
answers
162
views
calculating average word embedding Glove
I am trying to calculate Document embedding as average of word embedding. I use Glove word embedding. I have tokenized the documents.
def avg_word_emb(documents):
docs_array = np.zeros((len(...
0
votes
0
answers
22
views
on searching in milvus return single ID for list of vectors
I'm using milvus 2.1.2
I have data in this format:
{
ID : 001
Vec : { [0.01, 0.05, ...],
[0.03, 0.08, ...],
.
.
.
[0....
0
votes
3
answers
331
views
tensorflow UnknownError: Graph execution error: JIT compilation failed. [Op:__inference_restored_function_body_9127]
I was trying to use UNIVERSAL SENTENCE ENCODER from tensorflow hub.
Downloaded and extracted universal sentence encoder from hub
and when i tried to predict a senetence it showed an Error saying
...