Questions tagged [word-embedding]

For questions about word embedding, a language modelling technique in natural language processing. Questions can concern particular methods, such as Word2Vec, GloVe, FastText, etc, or word embeddings and their use in machine learning libraries in general.

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

Stuck with SVM classifier using word embeddings/torchtext in NLP task

I'm currently on an task where I need to use word_embedding feature, glove file and torchtext with SVM classifier. I have created a sperate function for it where this is what the implementation of ...
0 votes
2 answers
31 views

How does gensim calculate sentence embeddings when using a pretrained fasttext model?

According to this answer, sentence similarity for FastText is calculated with one of two ways (depending if the embeddings are created superviser or unsupervised) The mean of the normalized word ...
  • 4,405
0 votes
1 answer
10 views

Evaluate FastText embeddings

I want to evaluate my FastText model (trained on my own corpus). For semantic meaning I understand that we can use a dataset containing several pairs of two words which have been scored by humans, and ...
  • 4,405
-1 votes
0 answers
11 views

Predicting word embedding directly from softmax layer without argmax using machine learning model

I want to build a model such that which will predict the corresponding word2vec embedding directly from the softmax layer without taking argmax . How will be the model architecture looks like and what ...
1 vote
0 answers
15 views

Understanding Signal Representation and Processing using Continuous Bag of Words (CBOW) Model

I'm attempting to improve the representation of my signal at each time step using the CBOW method. However, my signal consists of 1000 time steps, which is too large to work with. To address this, I ...
  • 11
0 votes
2 answers
22 views

Topic modeling with word embeddings

I am currently trying to create an LDA model with word embeddings. Here is the code: from gensim.models.ldamodel import LdaModel from gensim.corpora.dictionary import Dictionary from gensim.test.utils ...
2 votes
1 answer
126 views

Semantic searching using Google flan-t5

I'm trying to use google flan t5-large to create embeddings for a simple semantic search engine. However, the generated embeddings cosine similarity with my query is very off. Is there something I'm ...
0 votes
0 answers
49 views

OutOfMemoryError when I create model embeddings

Just started learning huggingface transformers. I am trying to create embeddings of a large amount of text but I always run into outOfMemoryErrors. I am not sure what I am doing wrong. I am new to ...
0 votes
0 answers
28 views

running locale text embedding projector with Pytorch

I want to run a text embedding projector locally using Pytorch (not tensorflow). Everything is installed and running this comment bring up the TensolBoard empty: import keyword import torch meta = [] ...
  • 2,595
0 votes
0 answers
15 views

How to get vocabulary from WordEmbeddingsModel in sparknlp

I need to create an embedding matrix from embeddings generated by WordEmbeddingsModel in sparknlp. Until now i have this code : from sparknlp.annotator import * from sparknlp.common import * from ...
  • 31
1 vote
1 answer
62 views

What does the embedding elements stand for in huggingFace bert model?

Prior to passing my tokens through encoder in BERT model, I would like to perform some processing on their embeddings. I extracted the embedding weight using: from transformers import TFBertModel # ...
1 vote
0 answers
15 views

How to find Sentence Transformer support languages?

I want to get the sentence embedding results to find the sentence similarities in my NLP project. Since I am working with a low-resource language (Sinhala), I want to know whether any ...
  • 31
1 vote
1 answer
41 views

Embedding version control of the weaviate

In Weaviate, the vector engine, I wonder how this can handle version issue of embedding model. For instance, considering the (trained) word2vec model, embedded vectors from different models must be ...
  • 466
0 votes
0 answers
18 views

Text classification using NER, SRL, etc

I'm working on a task of classifying text complaints and I extracted some features like Named Entities, Events, Time Expressions, Semantic Role Labels, etc. I want to classify the text according to ...
  • 1
0 votes
0 answers
52 views

Julia/Flux creating a model correctly - using Chain Embedding layer reshaping & Dense layers

I am trying to follow an old article about word embedding in Julia language with Flux: https://spcman.github.io/getting-to-know-julia/deep-learning/nlp/flux-embeddings-tutorial-1/ As it is an old ...
  • 111
0 votes
0 answers
13 views

Running SVM on positional embeddings using keras for text classification

How can I run SVM on a large text classification dataset for detecting fake news of 400 thousand entries that uses positional encoding for embeddings from keras and has a maximum sentence length of 15 ...
  • 37
0 votes
1 answer
27 views

Can not run the code of this repository - NETL-Automatic-Topic-Labelling-

I am trying to run this code - Automatic Labelling of Topics with Neural Embeddings The problem is that they did not mention what versions they used for the libraries and tools they used. Sadly, not ...
  • 998
0 votes
1 answer
25 views

How to encode the shortest dependency path between words in sentence in neural network?

I am doing a first project on relation extraction between two entities in sentences in NLP. I use an LSTM model, and in addition to inputting the word embedding, I want to input the shortest ...
0 votes
0 answers
29 views

How to use gpu when computing word embeddings in textEmbed()?

I noticed that the textEmbed function has a device option to choose between cpu or gpu. But when running textEmbed with "device" set to "gpu" I get the message: "Unable to use ...
0 votes
0 answers
36 views

The most performant and proper way to get similarities between large embeddings/vectors

I have 2 data frames with 69 & ~230.000 rows and (KEY, Embedding) columns. The embedding columns are array type and each has a length of 768 which I obtained from a fine-tuned transformer model (...
0 votes
0 answers
14 views

any word embedding model for medical contexts?

Hey so I'm searching for a word embedding model trained on medical data to write a program that finds a medical diagnosis from free text. I've found a model called bioSentVec but I'm not sure that ...
  • 13
0 votes
0 answers
45 views

Including a covariate in a word embedding model in R using text2vec and quanteda packages

I am trying to build a word embedding model in r with the following code: library(quanteda) library(text2vec) fcm_ <- fcm(tokens, context = "window", count = "weighted", ...
0 votes
0 answers
14 views

Select Topic Words from Clusters

I am following this solution for clustering: https://towardsdatascience.com/clustering-contextual-embeddings-for-topic-model-1fb15c45b1bd For step four "Select Topic Words from Clusters", I ...
  • 333
0 votes
0 answers
38 views

SpaCy "en_core_web_trf" gives empty result

To use BERT for word embedding, why does SpaCy "en_core_web_trf" give an empty result?
  • 333
0 votes
0 answers
20 views

Tensorflow: Using Word Embeddings to Tranform Target Train into Embedding Vectors, does it make sense?

I have been studying TF for only a few months now, so bear with me, and thought of this as a hypothetical question. For example, I have as input, audio spectrograms of words, and I am training a model ...
0 votes
1 answer
42 views

Gensim Word2Vec produces different most_similar results through final epoch than end of training

I'm using gensim's Word2Vec for a recommendation-like task with part of my evaluation being the use of callbacks and the most_similar() method. However, I am noticing a huge disparity between the ...
  • 137
0 votes
0 answers
34 views

Text Classification roBERTA word embedding to CNN, accuracy not improving

I am learning CNN and roBERTa word embedding, I create a sentiment analysis with 3 label, -1 for negative, 0 for neutral, 1 for positive. I already have word embedding from roBERTa but when processing ...
1 vote
1 answer
44 views

Python / Word2Vec: How to project a word on two axis e.g. 'man-woman' and 'rich-poor'

How can I project a word W on an X-axis which represents the scale 'man-woman' and on Y-axis which represents the scale 'poor-rich'? Let's say my word is: word = model.wv['king'] and I would like to ...
  • 11
0 votes
1 answer
68 views

'Word2Vec' object has no attribute 'infer_vector'

This is the version of gensim I am using: Name: gensim Version: 4.3.0 Summary: Python framework for fast Vector Space Modelling Home-page: http://radimrehurek.com/gensim Author: Radim Rehurek Author-...
1 vote
1 answer
104 views

Calculate cosine similarity between 2 words using BERT

I am trying to calculate the cosine similarity beteween two given words using BERT, but I am getting an error which says: IndexError: Dimension out of range (expected to be in range of [-1, 0], but ...
  • 29
0 votes
0 answers
22 views

Why the ouput of nn.Embeddings(vocab_size, dim) chnages on re-running the code for same input string?

I am trying to understand how word embeddings are generated, I've been reading that 1-hot encoded vector is used and it servers as a lookup table but, I want to print that and see, how can I do that. ...
0 votes
0 answers
83 views

The range of similarity in openAI text-embeddings is not [0, 1]

I am using openAI Ada to create text-embeddings. When i calculate the similarity between these embeddings my answer is in the range [0.7, 0.9] approximately. In the rest of my project I would need ...
0 votes
0 answers
23 views

Problem with using multiple inputs with embedding layers in a Keras Model

I am trying to build a basic NN using Keras using multiple input layers, some of which are connected to Embedding layers. I am using the data for Kaggle's Housing Prices competition. Here is the ...
  • 41
0 votes
1 answer
183 views

How do I introduce a new subject without extra training in Stable Diffusion?

Suppose I have a dataset of 1000 pokemon, I have 10 images of Pikachu, 10 images of Bulbasaur, etc. I also have a metadata specifying the name of each pokemon exactly. So from the metadata, I can know ...
  • 3,639
0 votes
0 answers
83 views

Is normalization of word embeddings important?

I am doing actor-critic reinforcement learning for an environment that is best represented as a "bag-of-words". For this reason, I have opted to use a single body, multi-head approach for ...
0 votes
1 answer
46 views

Use three transformations (average, max, min) of pretrained embeddings to a single output layer in Pytorch

I have developed a trivial Feed Forward neural network with Pytorch. The neural network uses GloVe pre-trained embeddings in a freezed nn.Embeddings layer. Next, the embedding layer splits into three ...
  • 1,232
0 votes
1 answer
51 views

Changing shapes of PyTorch tensors and numpy arrays

I'm using CLIP model from huggingface to generate image embeddings, and I'm struggling with the output's shape. I'm trying to get a numpy array of shape (n, 512) - given n samples and 512 is the ...
  • 25
0 votes
0 answers
78 views

How to train string feature as word embedding vectors as part of a Linear Regression Tensorflow model?

How do I train string feature as word embedding vectors as part of a linear regression Tensorflow model? I found example online about how to uses Tensorflow to vectorize strings and use these vectors ...
0 votes
0 answers
56 views

While installing BioSentVec I am getting an error for the ```make``` command: The system cannot find the file specified

I am trying to install BioSentVec but getting an error while running the make command. I am following the instructions in the answers to this question. I will focus on the second answer, which is as ...
  • 1,059
0 votes
1 answer
75 views

How to work with Doc2Vec and which approach is better training the model on my dataset or using a pretrained model?

I am building a classification model for a dataset of items. Basically, I have 2 columns ex: Item name category unsalted butter dairy and eggs cheese dry grocery peanut butter cream dry grocery I ...
-1 votes
1 answer
67 views

Subword vector in fastText?

I can't figure out what a subword input vector is. I read in the newspaper that the subword is hashed, the subword is the hash code, hash code is a number, not a vector Ex: Input vector of word eating ...
  • 1
0 votes
0 answers
42 views

embeddings distribution wrong

I'm having the code below which is supposed to plot word embeddings. Since it creates a list of embeddings of identical words I would have expected a cluster of points - all embeddings at one point. ...
  • 1,803
3 votes
2 answers
379 views

Classic king - man + woman = queen example with pretrained word-embedding and word2vec package in R

I am really desperate, I just cannot reproduce the allegedly classic example of king - man + woman = queen with the word2vec package in R and any (!) pre-trained embedding model (as a bin file). I ...
  • 4,140
1 vote
1 answer
36 views

Generate and Read Word Embeddings with KeyedVectors

I trained a neural network with tensorflow and extracted the weights from the embedding layer to make an array of embeddings. I generated it as a txt file and I can't read it with KeyedVectors ...
0 votes
0 answers
102 views

combining dense (word embedding) and sparse vector (one hot encoding) but my kernel keeps crushing

The function combine_sparse_and_dense_features(dense_vectors, sparse_features) is to combining dense vectors (word embedding) and sparse features (one hot encoding). Dense vector array shape : (203621,...
  • 1
0 votes
0 answers
16 views

Where do w in Word2Vect come from?

can someone help me, please. Based on the image, I got a question where's the w1 and w2 actually comes from? I try to break the model down from SoftMax to predict, but I don't know how to find the w1 ...
0 votes
0 answers
74 views

How to get vector embedding of (lat, long) points or GPS trajectories?

I have a data that consists of approx. 1.5M taxi trips. Each of these trips have it's GPS trajectory traveled along as a feature. I would like to embed these trajectories, which are lists of (lat, ...
  • 13
0 votes
0 answers
162 views

calculating average word embedding Glove

I am trying to calculate Document embedding as average of word embedding. I use Glove word embedding. I have tokenized the documents. def avg_word_emb(documents): docs_array = np.zeros((len(...
0 votes
0 answers
22 views

on searching in milvus return single ID for list of vectors

I'm using milvus 2.1.2 I have data in this format: { ID : 001 Vec : { [0.01, 0.05, ...], [0.03, 0.08, ...], . . . [0....
0 votes
3 answers
331 views

tensorflow UnknownError: Graph execution error: JIT compilation failed. [Op:__inference_restored_function_body_9127]

I was trying to use UNIVERSAL SENTENCE ENCODER from tensorflow hub. Downloaded and extracted universal sentence encoder from hub and when i tried to predict a senetence it showed an Error saying ...

1
2 3 4 5
21