Questions tagged [pytorch]

PyTorch is an open-source deep learning framework and API that creates a Dynamic Computational Graph, which allows you to flexibly change the way your neural network behaves on the fly and is capable of performing automatic backward differentiation.

Filter by
Sorted by
Tagged with
0 votes
1 answer
12 views

Print intermediate gradient values during backward pass in Pytorch using hooks

I am trying to print the value of each of the intermediate gradients during backward pass of a model, using register backward hooks: class func_NN(torch.nn.Module): def __init__(self,) : ...
Newbie's user avatar
  • 1
-2 votes
0 answers
50 views

In Python, is it better to call a function multiple times with the same parameters except one, or to create a class with such function as method? [closed]

I have a project in python, about image reconstruction, where within an iterative procedure I have to call a function f at least twice. To summarize the important details, the procedure looks like ...
Eddymage's user avatar
  • 1,014
-1 votes
0 answers
18 views

Azure Docker image build cache past attempts? [closed]

I am currently trying to build a docker image that builds Pytorch with MSCCL backend, but I am not sure where I can achieve this. By where I mean in which environment as this is a memory and compute-...
ryupark1321's user avatar
0 votes
0 answers
32 views

Pytorch, When I use backward() in the loop, the message failed on the second loop

I use loops to propagate forward and backward through my model. However, I encounter an error notification during the second loop with the following message: RuntimeError: Trying to backward through ...
KikkiZhang's user avatar
0 votes
0 answers
17 views

How to get weights for the embedding layer after training using Pytorch's nn.Transformer?

I'm a biginner of NLP. Also, I am not an English speaker, so I apologize for my poor English. I recently try to train a model using Pytorch's nn.Transformer module to get weights for the embedding ...
sin's user avatar
  • 1
0 votes
0 answers
9 views

how AccumulatedGrad object's register_hook works?

class FFN(nn.Module): def __init__(self): super(FFN,self).__init__() self.linear1 = nn.Linear(4,20,bias=True) self.linear2 = nn.Linear(20,4,bias=True) def forward(self,...
baba zhou's user avatar
0 votes
0 answers
30 views

Pytorch RunTimeError: one of the variables needed for gradient computation has been modified by an inplace operation

I am currently working on a project that involves the implementation of Bayesian GAN. I am following the Pytorch implementation, and this seems to follow the style of the official tutorial of DCGAN. ...
Preetom Saha Arko's user avatar
0 votes
0 answers
28 views

Broken pytorch installation

I was trying to install pytorch, I went to the official website and copied the command for windows 10, pip, cuda 11.8, so it should be correct. My first installation failed because I had too little ...
Plush Loler's user avatar
0 votes
0 answers
30 views

how to resolve the error message: return TCPStore( ) RuntimeError: unmatched '}' in format string

In the past few days, I've been trying to run the code from this GitHub repository: improved-diffusion on my computer. In essence, it involves training a denoising diffusion probabilistic model, which ...
ChuckLin's user avatar
0 votes
0 answers
34 views

NaN values during pytorch training using BCEWithLogitsLoss

I am trying to reproduce the model from Malceiver: Perceiver with Hierarchical and Multi-modal Features for Android Malware Detection paper. During training, in the first epoch, at batch_idx == 81 I ...
Vlad-Sebastian Cretu's user avatar
-3 votes
0 answers
18 views

Comparing Accuracy: Single GPU vs. 8 GPUs [closed]

Would PyTorch yield different accuracy when executed on 8 GPUs (distrusted mode) compared to running on 1 GPU? Is it expected to observe variations in results? For instance, the accuracy on a single ...
External_Happy_77's user avatar
0 votes
1 answer
20 views

Is there a way to save the pytorch model for environment without pytorch package?

We are training a model for a small minimal dependency software. The developer refused to include the PyTorch in their independence. Is there a way to save the pytorch model in a way that they can ...
Chenxin's user avatar
  • 21
0 votes
2 answers
35 views

How to install chatterbot in google colab

I plan to install Chatterbot in Google Colab, but I have been unable to install it. I have been getting errors when using !pip install chatterbot directly. Below are screenshots of my code and the ...
Kevin Liang's user avatar
0 votes
0 answers
18 views

CUDA out of memory when using Optuna

Half month ago, I can use Optuna without a problem to do a 48-Hour study, with around 150+ trials. Yesterday I tried Optuna again on the same model, same dataset, same batch size and same device (A100 ...
Tianjian Qin's user avatar
0 votes
1 answer
31 views

RuntimeError: Given groups=1, weight of size [32, 3, 5, 5], expected input[1, 32, 3, 784] to have 3 channels, but got 32 channels instead

RuntimeError: Given groups=1, weight of size [32, 3, 5, 5], expected input[1, 32, 3, 784] to have 3 channels, but got 32 channels instead this is my code: class Conv(nn.Module): def __init__(self): ...
nguyen sang's user avatar
-4 votes
0 answers
59 views

What part of machine learning model training time takes actual matmul? [closed]

I'm learning PyTorch and want to understand all related stuff that goes under the hood when PyTorch trains neural network. I wonder about two things: where is computation graph stored while training? ...
Mateusz Woźniak's user avatar
-1 votes
0 answers
18 views

How to develop a pipeline of classifiers using Pytorch Lighning? [closed]

Let's say, I have developed a classifier, in order to classify between labels (Label 1 & Label 2), but the results are shown that the model can classify only class 1. Then, I want to use again the ...
ChackM's user avatar
  • 7
-1 votes
0 answers
15 views

HuggingFace - "ModuleNotFoundError" for diffusers.UniPCMultistepScheduler

new to HuggingFace and ml/dl in general, so just wanted to try a model qr-code-monster found a script that I can use as an example so just got started with it in colab, I am stuck at ...
Priyanshu Jha's user avatar
0 votes
3 answers
63 views

How can I read hdf5 files stored as 1-D array. and view them as images?

I have a large image classification dataset stored in the format .hdf5. The dataset has the labels and the images stored in the .hdf5 file. I am unable to view the images as they are store in form of ...
Aleph's user avatar
  • 223
-2 votes
1 answer
22 views

Need advice on a loss function for a pytorch surrogate model [closed]

I was hoping to get advice on implementing a loss function for a surrogate model. I am working with micromechanical simulations of gas pores produced during processing in Additive Manufacturing, and ...
ddiaz312's user avatar
0 votes
1 answer
30 views

Transformers: Cross Attention Tensor Shapes During Inference Mode

Having been trying to figure this out for a while. I found this similar question but I don't think the proposed answer actually addresses the question. During inference mode of an Encoder/Decoder ...
NickBraunagel's user avatar
0 votes
0 answers
17 views

RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported

When running my VAE in a jupyter notebook, I get this CUDA device-side assert error. It started appearing after changing my loss function, but after reverting all changes, I still get it. for e in ...
Ryan Baker's user avatar
0 votes
0 answers
17 views

PyTorch multiprocessing is filling up shared memory. Is there any way to manually invoke torch_shm_manager to garbage collect?

I'm trying to process input data in parallel so it can be loaded into a PyTorch dataset. However, when it reaches ~20,000 data points or so, the parallel processes crash claiming it can't allocate ...
Eric Bell's user avatar
1 vote
0 answers
17 views

How can I get a pytorch Tensor containing some other Tensor's size (or shape) without conversion to Python int?

In the context of exporting pytorch code to ONNX, I get this warning: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this ...
Dietmar's user avatar
  • 526
0 votes
1 answer
39 views

How do I calculate the loss when samples per batch have different shapes?

I have a training function like so: def training(): model.train() train_mae = [] progress = tqdm(train_dataloader, desc='Training') for batch_index, batch in enumerate(...
Vivek Joshy's user avatar
1 vote
1 answer
48 views

Change batch size using a list of Pytorch's data loader

During the training of my neural network model, I used a Pytorch's data loader to accelerate the training of the model. But instead of using a fixed batch size before updating the model's parameter, I ...
pyaj's user avatar
  • 557
0 votes
0 answers
21 views

For splitting the data, does pytorch do it by itself randomly if I don’t specify split ratio?

For splitting the data, does pytorch do it by itself randomly if I don’t specify split ratio? I am working on pytorch deep learning model autoencoder, i have dataset (csv file) with 10 classes and 500 ...
Veerbasant Reddy's user avatar
0 votes
1 answer
21 views

Error With Pytorch: "RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x150528 and 784x512)"

I am trying to train a model by Pytorch. The data set belongs to me and it has 2 folders: train_data and test_data. And every folder has 2 folders: detected and not_detected (these are my classes). ...
Baguviks's user avatar
0 votes
0 answers
21 views

How to fix "TypeError: code() argument 13 must be str, not int" in fine-tuning Yolov8

I am fine-tuning Yolov8 with custom dataset. The error always happen after the First epochs. This means that the model train for some time but after a finishing first epoch, the error popup. Here is ...
Nuntea7's user avatar
0 votes
0 answers
22 views

How to fix policy loss quick convergence (PPO)?

I am currently working on a RL agent that has to learn how to play pacman. I implemented a PPO algorithm. It worked fine on a small map with only a single pile to eat and one ghost. (The pile was ...
Hamidou's user avatar
  • 75
-1 votes
0 answers
29 views

Categorise a frame in a video to see if it contains one of a set of slides [closed]

I want to process videos where a lecturer stands in front of a whiteboard with multiple slides on it. They may move around, block the board, or write on a slide. I will have the collection of all ...
VeeBee's user avatar
  • 43
1 vote
0 answers
25 views

Pytorch Normalize() receiving torch.float32 tensor but recognising it as torch.int32

I'm trying to normalize the NYU v2 Depth Dataset, and this is the transform I'm applying to each image coming through: def standard_transform(normalise=False): composition = [ transforms....
Sam Y's user avatar
  • 13
-5 votes
0 answers
32 views

Should I use TensorFlow or PyTorch for my project? [closed]

I'm working on a mentor-mentee matching project where I aim to use deep learning techniques and language models like BERT or GPT to analyze textual profiles of mentors and mentees for suitable matches....
Ayush Kumar's user avatar
0 votes
0 answers
11 views

Gathering result using Distributed Data Parallel needs too much video memory

the following is my task method used for multi-GPU predicting. def worker(self, rank:int, pickle_file_name:str): torch.distributed.init_process_group('nccl', rank=rank, world_size=self.world_size) ...
landings's user avatar
  • 696
0 votes
0 answers
36 views

What is the Highest Version of PyTorch that can be Built from Source?

As most of you people already know, the official binary of PyTorch will not work if you have an older GPU. If your GPU has a Compute Capability of 3.0 or 3.5, then you are screwed. In order to use ...
TechVillain's user avatar
0 votes
0 answers
17 views

ValueError: The `index` argument must be one-dimensional (got 2 dimensions)

receiving the following error: ValueError: The `index` argument must be one-dimensional (got 2 dimensions) while using RGCN model of pyTorch: output = rgcn_layer(x = x, edge_index = edge_index, ...
Tasbiha's user avatar
  • 104
0 votes
0 answers
10 views

URLError: <urlopen error [Errno 110] Connection timed out> when I try to load the GTZAN dataset

I am trying to load the GTZAN dataset from PyTorch using torchaudio.datasets.GTZAN gtzan_dataset = torchaudio.datasets.GTZAN(root='./data', download=True) waveform, sample_rate, label = gtzan_dataset[...
xf4c70r's user avatar
0 votes
0 answers
16 views

PyTorch CNN Returns Only One Result After Training

I'm training a CNN image classifier. The network classifies 255 x 255 RGB images into five categories numbered 0 to 4. But the network is behaving strangely during training. Although the loss function ...
Paulo Mendes's user avatar
0 votes
1 answer
21 views

einops rearrange vs torch view

I am currently implementing the LoFTR model and came across the following code: feature_c0.shape -> torch.Size([1, 256, 60, 60]) rearrange(feature_c0, 'n c h w -> n (h w) c').shape -> torch....
Robin Müller's user avatar
0 votes
1 answer
29 views

PyTorch Convolutional Neural Network (CNN) Training Batch Size Mismatch Error

I have a dataset folder with 4 subfolder named contexts(forest,glacies,mountain,sea) each folder has approximately 25000 jpg image aboutcontext, I'm training a Convolutional Neural Network (CNN) in ...
Furkan Şenoğlu's user avatar
0 votes
0 answers
12 views

pytorch Slowdown in CPU-based Preprocessing After Loading Model Weights onto GPU

I'm encountering an unexpected issue in my machine learning pipeline. Specifically, after loading the model weights onto the GPU, the CPU-based preprocessing function, which relies solely on NumPy and ...
Henry38947's user avatar
0 votes
0 answers
12 views

My Pytorch multi-GPU prediction code freezes at `spawn()`

The following small code does multi-GPU prediction using Pytorch. I launch multiple tasks using torch.multiprocessing.spawn(). Inside task, I put no real prediction code. Just putting something into a ...
landings's user avatar
  • 696
0 votes
0 answers
13 views

Different behavior between pytorch.cdist and a self-made distance function between pair of points

For a loss I had to compute the distance between each pair of points in a point cloud. I did it using the following method in a first time (x is a tensor of shape (Batch, nb_points, 3)): dist = (x....
Rapoyo's user avatar
  • 1
0 votes
0 answers
27 views

torch.topk vs torch.tensor.max()

Has anyone tried comparing the values output by the above 2 functions ? I have a problem where i want to obtain the top 2 values in the dim 0 of a tensor. When i tried to compare the outputs of the ...
Ashwath S's user avatar
0 votes
1 answer
66 views

Use case for np.log2 vs math.log2

Sometimes, authors use np.log2 instead of math.log2. For example, in this PyTorch code: num_pools = int(np.log2(spatial)) (where spatial is a Python number) Because math.log2 is a built-in an ...
Richie Bendall's user avatar
0 votes
0 answers
10 views

Custom Pytorch Lightning metric for loss function is causing errors during metric.forward & update calls

PyTorch-Forecasting version: 1.0.0 PyTorch version: 2.1.0 Python version: 3.10.13 Torchmetrics version: 1.2.0 Operating System: Windows 10 Expected behavior I defined a custom class for a loss ...
Farooq Zahid's user avatar
0 votes
0 answers
26 views

I'm getting keyerror in python . "Exception has occurred: KeyError 'labels'"

In my python program I'm getting this error: Exception has occurred: KeyError 'labels' KeyError: 'labels' I'm getting error from this code: labels = data['labels'].reshape(-1).astype(np.int32) Can ...
Arghya Chatterjee's user avatar
0 votes
1 answer
15 views

Can't pickle local object 'get_tokenizer.<locals>.<lambda>'

--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) d:\abie\Coding\Tugas Akhir brow\VisualGPT-main\...
Aabie Nugraha's user avatar
0 votes
0 answers
41 views

Issue with DPO training on OpenLLAMA 3B V2 model: "I am sorry I cannot reply to that question" response during inference

Description: I am currently working on a project involving Direct Preference Optimization (DPO) training with the TRL library from Hugging Face. The goal is to align responses from the OpenLLAMA 3B V2 ...
Tatva Joshi's user avatar
0 votes
0 answers
23 views

What is the correct technique for incorporating access to large machine learning data sets in a Ray Tune Tuner() object?

I'm attempting to use Ray Tune 2.8.0 to optimize the hyperparameters of a pytorch neural network model--a fairly standard use case as far as I can understand. Pytorch is distributed with several ...
stachyra's user avatar
  • 4,463

1
2 3 4 5
464