Skip to main content
Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

TGI with Mixtral-8x7B CUDA no kernel image available error

I have a system with the following configuration: Nvidia Quadro RTX 8000 GPU with 48GB VRAM Nvidia SMI driver version 550.90.07 with CUDA version 12.4 nvidia-container-toolkit version 1.15.0 I'm ...
user8797614's user avatar
0 votes
1 answer
22 views

Embedding HuggingFaceModel on SageMaker with Llamaindex return 500

While trying to implement an HuggingFace Embedding using LLamaindex on Sagemaker, Sagemaker Endpoint return a Worker Died (error500).If some had the same error, or any idea ? The documents (txt files) ...
Celi28's user avatar
  • 111
0 votes
0 answers
20 views

How does the data splitting actually work in Multi GPU Inference for Accelerate when used in a batched inference setting?

I followed the code given in this github issue and this medium blog I ran the batched experiment with process = 1 and process=4 it gave me the result but I'm confused right now because I thought the ...
Deshwal's user avatar
  • 4,092
0 votes
0 answers
12 views

How to use Gradio Client to generate images

I'm using gradio_client (Python) to generate images using the API, and the official space has a GPU issue, so I used https://dudeicuf-black-forest-labs-flux-1-schnell.hf.space (the problem also ...
Isus Ramzy's user avatar
-1 votes
0 answers
17 views

huggingface_hub/file_download.py receives urllib3 ConnectTimeoutError

Try replicating the code for : https://github.com/ximinng/DiffSketcher.Do all the environment configuration operations successfully, but when trying to run the code to start the model, errors happened:...
Ran An's user avatar
  • 1
1 vote
0 answers
22 views

Multiprocessing for CPU-bound task (ML inference) in Jupyter Notebook

I'm working on Windows, on a Jupyter Notebook in VSCode. I have a list of sentences and I want to apply a Hugging Face model to each one. Each one takes like 1 second, so I figured I could accelerate ...
zest16's user avatar
  • 583
0 votes
0 answers
25 views

llama-index parallel ingestion is failing with token exceed error for embedding model deployed externally

I am using the below test code to try parallel llamaindex ingestion. The code is taken from the link: https://docs.llamaindex.ai/en/stable/examples/ingestion/parallel_execution_ingestion_pipeline/ The ...
Pvk Kolambkar's user avatar
-1 votes
0 answers
16 views

how to finetune VATT Model by google for videos, I am under a tight deadline [of <15-10-2024>]

i am currently working on a project for which i took a pretrained model by google named VATT, it can recognize videos texts and audio very well.so i took the challenge to fine tune this model for ...
Ayush Yadav's user avatar
-3 votes
0 answers
19 views

Issue Running OpenAI Inference on Phi-3 [closed]

we’re trying to run use the OpenAI Python library and HF Inference Endpoints to run inference on this model: Saturn-App/generator-phi-test The model is a fine-tuned version of Phi-3 but it’s not ...
Johaan Mannanal's user avatar
1 vote
0 answers
9 views

How can I enable logging on my huggingface model that I'm using an inference point (api) for?

The catch is that I'm not using the huggingface 'transformers' library, since i'm not running the models locally but am using an inference point to the model on HF (similar to how we use the Openai ...
Ketan Kunkalikar's user avatar
0 votes
0 answers
34 views

Get error in transformers.Trainer. TypeError: Object of type set is not JSON serializable

I want to fine-tune the model "FacebookAI/roberta-large" on disaster-tweets dataset. I have problem in transformers.Trainer and I got this error: TypeError ...
Zahra Reyhanian's user avatar
0 votes
0 answers
37 views

BERTopic RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0, details in the post

I am trying to use the BERTopic library with a custom text generation model using the transformers library. However, I am getting this RuntimeError. I have tried to specify the device as 0 (GPU) in ...
Shivam Tawari's user avatar
0 votes
0 answers
71 views

huggingface diffusers inference for flux in fp16

the Hugginface Flux documentation, links to this comment, describing how to run inference in fp16: https://github.com/huggingface/diffusers/pull/9097#issuecomment-2272292516 it says: FP16 ...
memical's user avatar
  • 2,469
0 votes
1 answer
27 views

Audio File Won't Download Properly From Huggingface Streaming Dataset

I'm running this code to stream the huggingface dataset mozilla-foundation/common_voice_17_0: language = "en" buffer_size = 100 streaming_dataset = load_dataset("mozilla-foundation/...
Bobby Miller's user avatar
0 votes
1 answer
27 views

Inference workflow in compile mode using transformers.pipeline()

I am trying to run an inference workflow of a Llama model in compile mode using transformers.pipeline(). I am using the following line of codes to run the inference workflow in compile mode: model = ...
Arunima Ghosh's user avatar
1 vote
0 answers
23 views

Iterating a Huggingface Dataset from disk using Generator seems broken. How to do it properly?

I have a strange behavior in HuggingFace Datasets. My minimal reproduction is as below. # main.py import datasets import numpy as np generator = np.random.default_rng(0) X = np.arange(1000) ds = ...
LudvigH's user avatar
  • 4,439
0 votes
1 answer
116 views

TypeError: Expected a Runnable, callable or dict.Instead got an unsupported type: <class 'list'>

I am experimenting with LLM development. Here is my code: import langchain, pydantic, transformers from langchain import HuggingFacePipeline from langchain_core.prompts import PromptTemplate from ...
Toly's user avatar
  • 3,159
0 votes
0 answers
41 views

How to create a custom model with Hugginface PreTrainedModel

I'm trying to create a simple model with the code below, taken almost directly from the documentation, and receiving an error import torch from transformers import PretrainedConfig, PreTrainedModel ...
Tobi's user avatar
  • 1
0 votes
0 answers
16 views

Unexpected memory usage when training transformer models with LoRA and quantization

I am training models using transformers, accelerate, bitsandbytes and PEFT. I recently got a second graphics card (RTX 3090) and am seeing a decent increase in training speed. However when ...
gazm2k5's user avatar
  • 480
-3 votes
0 answers
250 views

How to use a Flux Schnell model without CUDA on Silicon Metal devices?

When I try to run the FluxPipeline on a Jupyter notebook it gets stuck on the cudart assertion error, but I can't have a CUDA compatible torch on my M1 Pro mac. Is there a workaround for this? I had ...
bendemonium's user avatar
0 votes
0 answers
9 views

Not able to use the uploaded model in Hugginface

I am new to hugginface, i have uploaded a model and now trying to re-use the uploaded model like this model = AutoModel.from_pretrained('LijinDurairaj/resume_evaluation_model') but i am getting this ...
Lijin Durairaj's user avatar
0 votes
1 answer
60 views

OutOfMemoryError: CUDA out of memory while using compute_metrics function in Hugging Face Trainer

I'm encountering a CUDA out of memory error when using the compute_metrics function with the Hugging Face Trainer during model evaluation. My GPU is running out of memory while trying to compute the ...
KainnT's user avatar
  • 13
0 votes
0 answers
29 views

langchain huggingface hub invoking issue

I wanted to create a ReAct Agent in langchain, it always didn't work, and when I invoked the llm itself (not the agent) I found out that its response starts with the input and then with the response ...
lowpex  's user avatar
0 votes
1 answer
42 views

Error using LlmFactory with "TheBloke/OpenHermes-2.5-Mistral-7B-GGUF" Huggingface

I tried replicating a simple Python code to create a small LLM model. I have macOS M1 machine. I created a separate environment where I installed Pytorch and llama-cpp-python. The code: from llmflex ...
Toly's user avatar
  • 3,159
0 votes
0 answers
13 views

Apptainer permission issue

I'm trying to run the docker image of Huggingface inference using apptainer. The issue is that i could mount my local directory and the models can be downloaded from huggingface but when the server ...
Bhuvaneshwaran Ilanthirayan's user avatar
0 votes
0 answers
22 views

HuggingFace accelerate device error when running evaluation

I am running some experiments on a multi-GPU cluster, and I'm using accelerate. I'm trying to calculate some metrics after every batch iteration in the training dataloader. While the training code ...
M. Koopmans's user avatar
0 votes
0 answers
17 views

ValueError: Need either a `state_dict` or a `save_folder` containing offloaded weights while trying to run the transformer.pipeline for Llama-3.1-8B

I tried to run "meta-llama/Meta-Llama-3.1-8B-Instruct" model on MAC M2 but getting error while running below code: model = "meta-llama/Meta-Llama-3.1-8B-Instruct" tokenizer = ...
kamal Dwivedi's user avatar
0 votes
0 answers
69 views

Why did my Google Colab session crash while running the Llama model?

I am trying to use the meta-llama/Llama-2-7b-hf model and run it locally on my premises but the session crashed during the process. I am trying to use the meta-llama/Llama-2-7b-hf model and run it ...
riddhi's user avatar
  • 13
0 votes
1 answer
36 views

How to reinitialize from scratch GPT2 XL in HuggingFace?

I'm trying to confirm that my GPT-2 model is being trained from scratch, rather than using any pre-existing pre-trained weights. Here's my approach: Load the pre-trained GPT-2 XL model: I load a pre-...
Charlie Parker's user avatar
1 vote
1 answer
54 views

How to ask multiple choice questions using pipeline

I want to ask multiple choice questions to the Hugging Face transformer pipeline; however, there does not seem to be a good task choice for this. I am referencing this: qa_pipeline = pipeline("...
θ_enthusiast's user avatar
0 votes
0 answers
14 views

What value should the sequence_length parameter be when converting to TFLite

I am converting a model hosted by HuggingFace to a TensorFlow Lite model by using HuggingFace's Optimum exporter. In their documentation they use the following example: optimum-cli export tflite --...
A Dark Divided Gem's user avatar
0 votes
0 answers
29 views

Issue in Accessing Spacy Models from Hugging Face

I had uploaded my NER model trained through Spacy on huggingface. While trying to inference, I am facing this issue: pip install https://huggingface.co/JeswinMS4/en_resume_pipeline/resolve/main/...
Jeswin MS's user avatar
-1 votes
0 answers
83 views

How to switch the mirror of huggingface in the program

I am deploying the model IC Light locally, I have created the conda environment, installed the 3rd party libraries via pip and ran gradio_demo.py. however he is reporting an error. C:\Users\Gavin\...
GavinCQTD's user avatar
-1 votes
0 answers
25 views

Huggingface Autotrain fail

I am trying to Huggingface Autotrain. My data set: Below is my configuration: When I start training I get this error: What am i doing wrong? Do I have to mention categorical columns and numerical ...
Kanishka's user avatar
  • 265
0 votes
1 answer
44 views

huggingface train only new tokens embedding

I want to add new tokens to a huggingface model and train only their embeddings. How can I do that? There are some ways to train only part of a weight tensor (e.g. https://discuss.pytorch.org/t/how-...
Bipolo's user avatar
  • 83
0 votes
0 answers
35 views

Training a huggingface model on streaming data

Problem: Something is not correct with set up and args. The training hangs and does not complete the eval step. Background and Code: I'm finetuning a whisper model on Catalan Common Voice data using ...
jboat's user avatar
  • 1
0 votes
0 answers
34 views

Issue In Using create_sql_query_chain from LangChain with Mistral 7b v0.1

Output generated by LLM I am working on Mistral 7b v0.1 LLM. i am using APPEnd Point from huggingface × langchain. I have configures the model successfully. Next I am using the create_sql_query_chain ...
Marvi's user avatar
  • 1
0 votes
0 answers
32 views

Duplicating/cloning huggingface space

I'm interested in setting up an inference API, using huggingface. I'm following an article https://medium.com/@dahmanihichem01/mixtral-and-rest-api-turning-mixtral-8x7b-into-an-api-using-huggingface-...
user1592380's user avatar
  • 35.7k
0 votes
0 answers
100 views

HuggingFace: Efficient Large-Scale Embedding Extraction for DNA Sequences Using Transformers

I have a very large dataframe (60+ million rows) that I would like to use a transformer model to grab the embeddings for these rows (DNA sequences). Basically, this involves tokenizing first, then I ...
youtube's user avatar
  • 387
0 votes
0 answers
57 views

What happens when we define a BitsAndBytesConfig without enabling load in 4 or 8?

I have been experimenting with doing PEFT with LoRA on some HuggingFace models. After performing many experiments and documenting them, I noticed that I did not specify the type of quantization when ...
Alaa's user avatar
  • 541
0 votes
1 answer
785 views

Finding config.json for Llama 3.1 8B

I installed the Llama 3.1 8B model through Meta's Github page, but I can't get their example code to work. I'm running the following code in the same directory as the Meta-Llama-3.1-8B folder: import ...
MatthewScarpino's user avatar
0 votes
0 answers
65 views

What does the "AttributeError: 'NoneType' object has no attribute 'cget_managed_ptr'" mean?

I'm trying to train a model with very standard HF code I've used before: import os from transformers import Trainer, TrainingArguments, AutoModelForCausalLM, AutoTokenizer from datasets import ...
Charlie Parker's user avatar
0 votes
1 answer
77 views

Why Langchain HuggingFaceEmbeddings model dimension is not the same as stated on HuggingFace

I was using langchain HuggingFaceEmbeddings model: dunzhang/stella_en_1.5B_v5. When I look at https://huggingface.co/spaces/mteb/leaderboard, I can see that the model is 8192. But when I do len(...
Nicolas REY's user avatar
0 votes
1 answer
30 views

Elasticsearch RequestError(400) 'search_phase_execution_exception'

I have a simple python app that uses Elasticsearch to store documents for Pokemon using this mapping: { "mappings": { "properties": { "id": { "...
KZiovas's user avatar
  • 4,509
0 votes
1 answer
114 views

RunnableSequence instead of LLMChain throws an error (updating from depreacated langchain)

When I have this code first from langchain_community.llms import HuggingFacePipeline from transformers import AutoTokenizer import transformers import torch model="meta-llama/Llama-2-7b-chat-hf&...
KansaiRobot's user avatar
  • 9,375
3 votes
1 answer
133 views

HuggingFace LLM Evaluate: RuntimeError: isin_Tensor_Tensor_out only works on floating types on MPS for pre MacOS_14_0. Received dtype: Long

Context: I tried to create an evaluation pipeline for a text summary task using HuggingFace evaluate packages. I got the issue of receiving dtype Long for the tensor, but I did not feed any long type ...
Duong Vu's user avatar
  • 182
0 votes
0 answers
15 views

How to Enforce Context-Only Responses in a Conversational Retrieval Chain with LangChain and OpenAI GPT-4?

I'm working on a Django project where I need to create a conversational retrieval system using LangChain with OpenAI's GPT-4 model. The goal is to ensure that the model only answers questions based on ...
Ejaz Habib Dar's user avatar
0 votes
0 answers
18 views

How can we implement LIM models from huggingface using langchain_huggingface library?

Langchain_Huggingface released recently in May 2024.I implemented some open source LLM using the same library (langchain_huggingface). That worked well. But now I am trying to generate images but ...
yameen_004's user avatar
0 votes
0 answers
24 views

Why loading llama2 takes about 30 GB of GPU RAM?

I am trying to finetune llama2 on a self-created dataset. The text in the dataset is quite long, so I used rope_scaling=8 for llama initialization. I also used 4-bit and qlora to save memory. However, ...
Rain Gu's user avatar
0 votes
0 answers
129 views

SSLError: HTTPSConnectionPool(host='huggingface.co', port=443)

I have the following script: test.py: from transformers import AutoTokenizer checkpoint = "kevinscaria/joint_tk-instruct-base-def-pos-neg-neut-combined" tokenizer = AutoTokenizer....
zest16's user avatar
  • 583

1
2 3 4 5
21