1,020
questions
0
votes
0
answers
5
views
TGI with Mixtral-8x7B CUDA no kernel image available error
I have a system with the following configuration:
Nvidia Quadro RTX 8000 GPU with 48GB VRAM
Nvidia SMI driver version 550.90.07 with CUDA version 12.4
nvidia-container-toolkit version 1.15.0
I'm ...
0
votes
1
answer
22
views
Embedding HuggingFaceModel on SageMaker with Llamaindex return 500
While trying to implement an HuggingFace Embedding using LLamaindex on Sagemaker, Sagemaker Endpoint return a Worker Died (error500).If some had the same error, or any idea ?
The documents (txt files) ...
0
votes
0
answers
20
views
How does the data splitting actually work in Multi GPU Inference for Accelerate when used in a batched inference setting?
I followed the code given in this github issue and this medium blog
I ran the batched experiment with process = 1 and process=4 it gave me the result but I'm confused right now because I thought the ...
0
votes
0
answers
12
views
How to use Gradio Client to generate images
I'm using gradio_client (Python) to generate images using the API, and the official space has a GPU issue, so I used https://dudeicuf-black-forest-labs-flux-1-schnell.hf.space (the problem also ...
-1
votes
0
answers
17
views
huggingface_hub/file_download.py receives urllib3 ConnectTimeoutError
Try replicating the code for : https://github.com/ximinng/DiffSketcher.Do all the environment configuration operations successfully, but when trying to run the code to start the model, errors happened:...
1
vote
0
answers
22
views
Multiprocessing for CPU-bound task (ML inference) in Jupyter Notebook
I'm working on Windows, on a Jupyter Notebook in VSCode.
I have a list of sentences and I want to apply a Hugging Face model to each one. Each one takes like 1 second, so I figured I could accelerate ...
0
votes
0
answers
25
views
llama-index parallel ingestion is failing with token exceed error for embedding model deployed externally
I am using the below test code to try parallel llamaindex ingestion. The code is taken from the link: https://docs.llamaindex.ai/en/stable/examples/ingestion/parallel_execution_ingestion_pipeline/
The ...
-1
votes
0
answers
16
views
how to finetune VATT Model by google for videos, I am under a tight deadline [of <15-10-2024>]
i am currently working on a project for which i took a pretrained model by google named VATT, it can recognize videos texts and audio very well.so i took the challenge to fine tune this model for ...
-3
votes
0
answers
19
views
Issue Running OpenAI Inference on Phi-3 [closed]
we’re trying to run use the OpenAI Python library and HF Inference Endpoints to run inference on this model: Saturn-App/generator-phi-test
The model is a fine-tuned version of Phi-3 but it’s not ...
1
vote
0
answers
9
views
How can I enable logging on my huggingface model that I'm using an inference point (api) for?
The catch is that I'm not using the huggingface 'transformers' library, since i'm not running the models locally but am using an inference point to the model on HF (similar to how we use the Openai ...
0
votes
0
answers
34
views
Get error in transformers.Trainer. TypeError: Object of type set is not JSON serializable
I want to fine-tune the model "FacebookAI/roberta-large" on disaster-tweets dataset. I have problem in transformers.Trainer and I got this error:
TypeError ...
0
votes
0
answers
37
views
BERTopic RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0, details in the post
I am trying to use the BERTopic library with a custom text generation model using the transformers library. However, I am getting this RuntimeError. I have tried to specify the device as 0 (GPU) in ...
0
votes
0
answers
71
views
huggingface diffusers inference for flux in fp16
the Hugginface Flux documentation, links to this comment, describing how to run inference in fp16:
https://github.com/huggingface/diffusers/pull/9097#issuecomment-2272292516
it says:
FP16 ...
0
votes
1
answer
27
views
Audio File Won't Download Properly From Huggingface Streaming Dataset
I'm running this code to stream the huggingface dataset mozilla-foundation/common_voice_17_0:
language = "en"
buffer_size = 100
streaming_dataset = load_dataset("mozilla-foundation/...
0
votes
1
answer
27
views
Inference workflow in compile mode using transformers.pipeline()
I am trying to run an inference workflow of a Llama model in compile mode using transformers.pipeline(). I am using the following line of codes to run the inference workflow in compile mode:
model = ...
1
vote
0
answers
23
views
Iterating a Huggingface Dataset from disk using Generator seems broken. How to do it properly?
I have a strange behavior in HuggingFace Datasets. My minimal reproduction is as below.
# main.py
import datasets
import numpy as np
generator = np.random.default_rng(0)
X = np.arange(1000)
ds = ...
0
votes
1
answer
116
views
TypeError: Expected a Runnable, callable or dict.Instead got an unsupported type: <class 'list'>
I am experimenting with LLM development.
Here is my code:
import langchain, pydantic, transformers
from langchain import HuggingFacePipeline
from langchain_core.prompts import PromptTemplate
from ...
0
votes
0
answers
41
views
How to create a custom model with Hugginface PreTrainedModel
I'm trying to create a simple model with the code below, taken almost directly from the documentation, and receiving an error
import torch
from transformers import PretrainedConfig, PreTrainedModel
...
0
votes
0
answers
16
views
Unexpected memory usage when training transformer models with LoRA and quantization
I am training models using transformers, accelerate, bitsandbytes and PEFT. I recently got a second graphics card (RTX 3090) and am seeing a decent increase in training speed. However when ...
-3
votes
0
answers
250
views
How to use a Flux Schnell model without CUDA on Silicon Metal devices?
When I try to run the FluxPipeline on a Jupyter notebook it gets stuck on the cudart assertion error, but I can't have a CUDA compatible torch on my M1 Pro mac. Is there a workaround for this?
I had ...
0
votes
0
answers
9
views
Not able to use the uploaded model in Hugginface
I am new to hugginface, i have uploaded a model and now trying to re-use the uploaded model like this
model = AutoModel.from_pretrained('LijinDurairaj/resume_evaluation_model')
but i am getting this ...
0
votes
1
answer
60
views
OutOfMemoryError: CUDA out of memory while using compute_metrics function in Hugging Face Trainer
I'm encountering a CUDA out of memory error when using the compute_metrics function with the Hugging Face Trainer during model evaluation. My GPU is running out of memory while trying to compute the ...
0
votes
0
answers
29
views
langchain huggingface hub invoking issue
I wanted to create a ReAct Agent in langchain, it always didn't work, and when I invoked the llm itself (not the agent) I found out that its response starts with the input and then with the response
...
0
votes
1
answer
42
views
Error using LlmFactory with "TheBloke/OpenHermes-2.5-Mistral-7B-GGUF" Huggingface
I tried replicating a simple Python code to create a small LLM model.
I have macOS M1 machine.
I created a separate environment where I installed Pytorch and llama-cpp-python. The code:
from llmflex ...
0
votes
0
answers
13
views
Apptainer permission issue
I'm trying to run the docker image of Huggingface inference using apptainer. The issue is that i could mount my local directory and the models can be downloaded from huggingface but when the server ...
0
votes
0
answers
22
views
HuggingFace accelerate device error when running evaluation
I am running some experiments on a multi-GPU cluster, and I'm using accelerate. I'm trying to calculate some metrics after every batch iteration in the training dataloader. While the training code ...
0
votes
0
answers
17
views
ValueError: Need either a `state_dict` or a `save_folder` containing offloaded weights while trying to run the transformer.pipeline for Llama-3.1-8B
I tried to run "meta-llama/Meta-Llama-3.1-8B-Instruct" model on MAC M2 but getting error while running below code:
model = "meta-llama/Meta-Llama-3.1-8B-Instruct"
tokenizer = ...
0
votes
0
answers
69
views
Why did my Google Colab session crash while running the Llama model?
I am trying to use the meta-llama/Llama-2-7b-hf model and run it locally on my premises but the session crashed during the process.
I am trying to use the meta-llama/Llama-2-7b-hf model and run it ...
0
votes
1
answer
36
views
How to reinitialize from scratch GPT2 XL in HuggingFace?
I'm trying to confirm that my GPT-2 model is being trained from scratch, rather than using any pre-existing pre-trained weights. Here's my approach:
Load the pre-trained GPT-2 XL model: I load a pre-...
1
vote
1
answer
54
views
How to ask multiple choice questions using pipeline
I want to ask multiple choice questions to the Hugging Face transformer pipeline; however, there does not seem to be a good task choice for this.
I am referencing this:
qa_pipeline = pipeline("...
0
votes
0
answers
14
views
What value should the sequence_length parameter be when converting to TFLite
I am converting a model hosted by HuggingFace to a TensorFlow Lite model by using HuggingFace's Optimum exporter.
In their documentation they use the following example:
optimum-cli export tflite --...
0
votes
0
answers
29
views
Issue in Accessing Spacy Models from Hugging Face
I had uploaded my NER model trained through Spacy on huggingface. While trying to inference, I am facing this issue:
pip install https://huggingface.co/JeswinMS4/en_resume_pipeline/resolve/main/...
-1
votes
0
answers
83
views
How to switch the mirror of huggingface in the program
I am deploying the model IC Light locally, I have created the conda environment, installed the 3rd party libraries via pip and ran gradio_demo.py. however he is reporting an error.
C:\Users\Gavin\...
-1
votes
0
answers
25
views
Huggingface Autotrain fail
I am trying to Huggingface Autotrain. My data set:
Below is my configuration:
When I start training I get this error:
What am i doing wrong? Do I have to mention categorical columns and numerical ...
0
votes
1
answer
44
views
huggingface train only new tokens embedding
I want to add new tokens to a huggingface model and train only their embeddings. How can I do that?
There are some ways to train only part of a weight tensor (e.g. https://discuss.pytorch.org/t/how-...
0
votes
0
answers
35
views
Training a huggingface model on streaming data
Problem:
Something is not correct with set up and args. The training hangs and does not complete the eval step.
Background and Code:
I'm finetuning a whisper model on Catalan Common Voice data using ...
0
votes
0
answers
34
views
Issue In Using create_sql_query_chain from LangChain with Mistral 7b v0.1
Output generated by LLM
I am working on Mistral 7b v0.1 LLM. i am using APPEnd Point from huggingface × langchain.
I have configures the model successfully.
Next I am using the create_sql_query_chain ...
0
votes
0
answers
32
views
Duplicating/cloning huggingface space
I'm interested in setting up an inference API, using huggingface. I'm following an article https://medium.com/@dahmanihichem01/mixtral-and-rest-api-turning-mixtral-8x7b-into-an-api-using-huggingface-...
0
votes
0
answers
100
views
HuggingFace: Efficient Large-Scale Embedding Extraction for DNA Sequences Using Transformers
I have a very large dataframe (60+ million rows) that I would like to use a transformer model to grab the embeddings for these rows (DNA sequences). Basically, this involves tokenizing first, then I ...
0
votes
0
answers
57
views
What happens when we define a BitsAndBytesConfig without enabling load in 4 or 8?
I have been experimenting with doing PEFT with LoRA on some HuggingFace models. After performing many experiments and documenting them, I noticed that I did not specify the type of quantization when ...
0
votes
1
answer
785
views
Finding config.json for Llama 3.1 8B
I installed the Llama 3.1 8B model through Meta's Github page, but I can't get their example code to work. I'm running the following code in the same directory as the Meta-Llama-3.1-8B folder:
import ...
0
votes
0
answers
65
views
What does the "AttributeError: 'NoneType' object has no attribute 'cget_managed_ptr'" mean?
I'm trying to train a model with very standard HF code I've used before:
import os
from transformers import Trainer, TrainingArguments, AutoModelForCausalLM, AutoTokenizer
from datasets import ...
0
votes
1
answer
77
views
Why Langchain HuggingFaceEmbeddings model dimension is not the same as stated on HuggingFace
I was using langchain HuggingFaceEmbeddings model: dunzhang/stella_en_1.5B_v5.
When I look at https://huggingface.co/spaces/mteb/leaderboard, I can see that the model is 8192.
But when I do
len(...
0
votes
1
answer
30
views
Elasticsearch RequestError(400) 'search_phase_execution_exception'
I have a simple python app that uses Elasticsearch to store documents for Pokemon using this mapping:
{
"mappings": {
"properties": {
"id": {
"...
0
votes
1
answer
114
views
RunnableSequence instead of LLMChain throws an error (updating from depreacated langchain)
When I have this code first
from langchain_community.llms import HuggingFacePipeline
from transformers import AutoTokenizer
import transformers
import torch
model="meta-llama/Llama-2-7b-chat-hf&...
3
votes
1
answer
133
views
HuggingFace LLM Evaluate: RuntimeError: isin_Tensor_Tensor_out only works on floating types on MPS for pre MacOS_14_0. Received dtype: Long
Context:
I tried to create an evaluation pipeline for a text summary task using HuggingFace evaluate packages. I got the issue of receiving dtype Long for the tensor, but I did not feed any long type ...
0
votes
0
answers
15
views
How to Enforce Context-Only Responses in a Conversational Retrieval Chain with LangChain and OpenAI GPT-4?
I'm working on a Django project where I need to create a conversational retrieval system using LangChain with OpenAI's GPT-4 model. The goal is to ensure that the model only answers questions based on ...
0
votes
0
answers
18
views
How can we implement LIM models from huggingface using langchain_huggingface library?
Langchain_Huggingface released recently in May 2024.I implemented some open source LLM using the same library (langchain_huggingface). That worked well. But now I am trying to generate images but ...
0
votes
0
answers
24
views
Why loading llama2 takes about 30 GB of GPU RAM?
I am trying to finetune llama2 on a self-created dataset. The text in the dataset is quite long, so I used rope_scaling=8 for llama initialization. I also used 4-bit and qlora to save memory. However, ...
0
votes
0
answers
129
views
SSLError: HTTPSConnectionPool(host='huggingface.co', port=443)
I have the following script:
test.py:
from transformers import AutoTokenizer
checkpoint = "kevinscaria/joint_tk-instruct-base-def-pos-neg-neut-combined"
tokenizer = AutoTokenizer....