Questions tagged [training-data]
A training set is a set of data used to discover potentially predictive relationships, used in fields like artificial intelligence, machine learning, and statistics.
1,590
questions
0
votes
0
answers
25
views
IndexError While training a model
While training a sign language translation pre-developed code from GitHub for demonstration. While training a model, received a error.
Here,
num_of_classes = 7
Input_size=(50,50)
The train image ...
0
votes
1
answer
13
views
Change all images in training set
I have a convolutional neural network. And I wanted to train it on images from the training set but first they should be wrapped with my function change(tensor, float) that takes in a tensor/image of ...
0
votes
1
answer
32
views
Can you train your set on a separate dataframe than the one you use to test your set with? (Python)
for a school project I need to make a forecast of the baseline sales. I've splitted the dataset into a X and Y set. X = all my variables except Total units (baseline), Y = Total Units (baseline).
I ...
0
votes
0
answers
10
views
Is it possible to supply list as feature in keras?
I have dateset like this
A B C D E
1 [1,2,3,4] [1,2] 0 4
3 [2,1,6,2] [4,3] 1 9
8 [3,6,9,1] [8,1] 4 5
---------------------
---------------------
Can I use this data to train the model in ...
-2
votes
0
answers
6
views
split ogg audio files into chunks for further training to detect birds
have a few minutes or up to 40 seconds audio clips (ogg). wanting to split them into 5 seconds chunks and keep the label. example: parrot - clip is 30 seconds, need it to then have 6, 5 second audio ...
2
votes
0
answers
11
views
How to split a dataset into 3: train, test and validation with ImageDataGenerator for my dataset from directory?
I want to split my image dataset into three parts: train=80, validation=10, and test=10 with ImageDataGenerator for my dataset from directory
train_dataset = image_generator.flow_from_directory(...)
...
0
votes
1
answer
15
views
Is there a python function for splitting the dataset?
I am trying figure out how I can split my train and test set in a way that, one of the features' mean, for ex. price will be constant for both train and test data. How can I do it? Thanks!
0
votes
0
answers
32
views
tensorflow python framework.errors_impl.UnimplementedError: Graph execution error:
I have a problem while executing the model.fit() line in my python program. I got the following error on executing it. (Sorry for the bad format, I am new here). I don't know, where exactly I messed ...
0
votes
0
answers
7
views
How to create a high quality wake-word solution for Android/ios app. Which technology stacks to try?
I am trying using:
Tensor flow lite- unable to achieve desired accuracy & training sample size requirement seems to be very high.
Q1) What's the minimum sample size required?
Kaldi - size send ...
0
votes
1
answer
14
views
how to add custom field to annotation in Yolo?
I want to use the kitti dataset and training it with the yolo algorithm.
In general, the format of the number of objects and the coordinates of the object in an image for each object are as follows:
&...
0
votes
0
answers
6
views
how to output metrics of both train and valid with fastai
I'm trying to use CSVLogger to log metrics during training. Although I see loss of both train and valid, I see only the metrics (e.g. accuracy) of train in the saved log file at each epoch:
Howe can ...
-1
votes
0
answers
13
views
ValueError: Input 0 of layer "sequential" is incompatible with the layer: expected shape=(None, 32, 32, 1), found shape=(None, 1, 32, 32, 3, 1, 1)
I'm trying to train the model I created:
model = Sequential()
model.add(Conv2D(8, (5, 5), activation="relu", input_shape=(32, 32, 1)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(...
0
votes
0
answers
13
views
How to train an LSTM on multiple independent time-series of sensor-data
I have sensor measurements for 10 different people performing the same experiment in which they need to complete a specific task. For each timestep in the measurements I have the corresponding label ...
0
votes
0
answers
12
views
Keras tensorflow with R stops after some epochs
Working with Rscript vers 4.2.0 and tensorflow 2.0.0. After some epochs, the training freezes. Any idea the cause of that issue?
batch_size <- 256
# Fit data into model
history <- model_big %>...
0
votes
1
answer
47
views
Pausing the training of a neural network - is it possible?
I am training a neural network and the new datasets I am training it on are really big in comparison to the ones used before. Since I am also saving the time it takes to end the training, I cannot use ...
0
votes
0
answers
8
views
VS Code Studio Error: assert batch_size <= self._num_examples AssertionError
Help me solve this error, I am trying to fit the train data, I don't understand these errors. Please help if anyone knows them.
Here is the link
https://github.com/thiagortk/Road-Surface-...
1
vote
0
answers
12
views
How to collect and label my own custom dataset to train OCR model?
I want to create my own OCR model to detect text which is of very small font size, so for doing that what is the procedure to collect the data and labeling. And after that how to train the OCR model?
0
votes
0
answers
43
views
PyTorch returns a "Dataset Not found" error
I am trying to train a dataset locally using yolov5 but it keeps throwing "Dataset not found" exceptions I also tried changing the paths to absolute paths but it still throws the exception.
...
0
votes
1
answer
28
views
Data type preference for training CNN?
I originally was using input data of int8 type ranging from 0-255 before learning that standardizing and normalizing should increase learning speeds and accuracy. I attempted both, with and without a ...
0
votes
1
answer
19
views
Training set and ARIMA plot isn't working?
I'm trying to plot my training data using ARIMA with a dataset that has the dates from 2021/02/28 - 2022/03/22. But when I plot my training ARIMA, it plots 2021-2022, leaves a gap in the graph and ...
0
votes
0
answers
37
views
Can't plot my time series data and training data on the same graph, with different colours?
Im trying to plot both the original data and my training data on the same graph with this code:
timeseries <- ts(df$new_cases_per_million, start=c(2021,2), end=c(2022,3), frequency=52)
ts_train <...
1
vote
1
answer
32
views
Do you understand why the Outputs of my YOLOV4 model has this shape? [duplicate]
I have a question.
I trained a YOLOV4 model for face detection and when i tried to look at the output on Neutron i found that the Bounding Box shape is [1,1,1] and Feature has the shape of [1,1,4] is ...
0
votes
0
answers
10
views
How do I filter outliers in test data based on z-scores of train data?
I have a train and test dataset. On the train dataset I detected and deleted outlier values, when their standard deviation is 5 times greater from the mean. If a z-score returned is larger than that, ...
-3
votes
1
answer
40
views
How to incorporate the validation set in machine learning?
I am trying to learn about machine learning, and I am having trouble understanding when and how to use the validation set. I have understood that it is used to evaluate the candidate models, before ...
0
votes
1
answer
18
views
IBM Watson NLU gives a "training validation failed. Too few samples for label class. Minimum of 5 required" Error
I am trying to train my model in IBM Watson NLU. I have to classify emails into 2 labels. The instance has been created and I am passing the correct Model ID too.
model_id = model['model_id']
...
0
votes
1
answer
29
views
Pytorch error mat1 and mat2 shapes cannot be multiplied
I receive this error. Whereas the size of my input image is [3072,2,2], so I flatten the image by the following code, however, I received this error:
mat1 and mat2 shapes cannot be multiplied (6144x2 ...
0
votes
0
answers
28
views
Training data in sentiment analysis
I'm doing sentiment analysis of tweets related to recent acquisition of Twitter by Elon Musk. I have a corpus of 10 000 tweets and I'd like to use machine learning methods using models like SVM and ...
0
votes
1
answer
40
views
Every time I train my CNN on matlab, is it remembering the old weights from the previous time I trained it? Or does it reset them?
So for example, I have trained a CNN on my data using a learning rate of 0.0003 and 10 epochs, with a minibatch size of 32. After training it, lets say I get an accuracy of 0.7. Now I want to adjust ...
-1
votes
1
answer
25
views
Is this a valid approach to scale your target in machine learning without leaking information? [closed]
Consider a housing price dataset, where the goal is to predict the sale price.
I would like to do this by predicting the "Sale price per Squaremeter" instead,
since it yields better results.
...
0
votes
1
answer
35
views
How to correctly generate training data based on percentages?
I have a question.
I am currently generating training data for my bayesian network as follows:
(also as code down below)
-> infected stands for people who are infected (0= not infected, 1= ...
0
votes
1
answer
20
views
Training loop doesn't start when num_workers >1 in the DataLoader
I have an NLP classification problem where I have a DataLoader object and its code is
train_patentload = DataLoader(train_patentset, batch_size=4, shuffle=True,num_workers=2)
When I run the training ...
-1
votes
0
answers
42
views
TensorFlow Keras sequential fit(x_train, y_train,...) incompatible with layer error
im getting a error then trying to train this model
its a multi layer input and 2 outputs
should I post the csv?
I tried to fix it by changing
model.compile(optimizer='adam', loss='binary_crossentropy',...
0
votes
0
answers
15
views
the effect of hyperparameters in the learning process
why do I get the same results after training even if I change only one hyperparameter?
so I decide to change more than one, but even when I change the number of iterations of training, the batch size, ...
1
vote
0
answers
25
views
How do I retrain BERT model with new data
I have already trained a bert model and saved it in the .pb format and I want to retrain the model with new datasets that i custom made, so in order to not to lose the previous training and such, how ...
0
votes
0
answers
16
views
.spacy training data to be used in Python - moved from JSON
I was using Spacy in Python with the training and testing datasets in JSON.
I was able to convert those to the .spacy format using the convert command line.
The problem I am facing now is that the old ...
0
votes
0
answers
16
views
how to train this code of extracting frames from video
import cv2
import os
vid = cv2.VideoCapture("D:\Huawei Share\Backup/vid1.mp4")
currentframe = 0
if not os.path.exists("data"):
os.makedirs("data")
while (True):
...
0
votes
0
answers
21
views
How to get RSquared value using Linear Regression smf.ols on Train/Test Data
I would like to adjust this code to add a tranformed regression equation made using smf.ols
Here is my current code:
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn....
0
votes
1
answer
29
views
High val_loss and low val_accuracy when training ResNet50 model
Ive been training a ResNet50 model with some added layers of my own, however with each epoch comes a higher val_loss and the same val_accuracy. I think its overfitting the model but not sure how i ...
1
vote
0
answers
38
views
Tesseract OCR Can't create .traineddata
The Problem:
I followed the step by step tutorial provided here to train my tesseract ocr for a new font. But on step 5 and 6 not all needed files are created.
What I did:
My image file is: en.va.exp0....
0
votes
1
answer
48
views
Train accuracy of Tensorflow model remains at zero
i build a model based on this architecture to make a binary classification ["0" for "Delay-insensitive", "1" for "Interactive"] using 5 features. The target ...
-1
votes
1
answer
30
views
Interpreting loss and metric curve
I am trying to train Unet model with the following parameters:
droput_: 0.2,
activation_: sigmoid,
activation_inner_: relu,
learning_rate_: 0.0001,
epsilon_: 1e-07,
loss : mse,
metric : rmse,
...
0
votes
0
answers
16
views
TfidfVectorizer Vectorizing words for train model
I'm preparing data for text classification models and I use TfidfVectorizer for vectorizing words.
I have 10k sentences and 47k vectorized words, so every sentence gets a list item of 47k vectorized ...
0
votes
0
answers
14
views
How can I predict a result from new data with a trained existing model in python?
'''
#Defining Model Function
def models(X_train, Y_train):
log = LogisticRegression(random_state=0)
log.fit(X_train, Y_train)
tree = DecisionTreeClassifier(criterion='entropy', ...
1
vote
2
answers
51
views
How can I split train and test data based on some conditions?
How can I split train and test data based on some conditions for the machine learning models? The test data should include the same spatial areas (x-y) for each year. Namely, I don't want the same ...
0
votes
1
answer
21
views
Convolutional neural network throws ErrorTypeError: __init__() takes 1 positional argument but 2 were given
I am passing a batch of 256 images x_s with the following dimensions [256, 3, 560, 448]. However I whenever i try to feed my images to the CNN i get the following error:
ErrorTypeError: __init__() ...
0
votes
0
answers
33
views
could not broadcast input array from shape (13,7,7,512) into shape (4,7,7,512)
i keep having this error and couldn't find a solution
ValueError: could not broadcast input array from shape (13,7,7,512) into shape (4,7,7,512)
and this is the code that i'm using for features ...
0
votes
0
answers
31
views
OpenCV Using a model to predict if a image belongs to a collection
The goal is to compare an image and select what the image contains from a selection of 5 other collections of images. Both the compare image and collection of images the image object takes up 100% of ...
0
votes
0
answers
19
views
Classify on test data, number of instances shows of training data
I have a dataset and I divided the dataset into training set(70%) and test set(30%). The total number of instances in my dataset is 1816. Training set has 1273 instances and test set has 543 instances....
0
votes
0
answers
35
views
Using a dictionary of multidimensional data for training
I would like to train my XGBoost model using a dictionary as training data, containing 5 keys and each key having windows of an array of data. Here is the figure explaning it:
I cannot give this ...
0
votes
0
answers
49
views
ValueError: multi_class must be in ('ovo', 'ovr') ... how to solve this error message
lr=LogisticRegression(random_state = 42, solver = 'newton-cg', max_iter = 200)
lr.fit(X_train, y_train)
y_valid_preds = lr.predict_proba(X_valid)[:,1]
print('Metrics for Validation data:')
...