Questions tagged [data-mining]

Data mining is the process of analyzing large amounts of data in order to find patterns and commonalities.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
16 views

Want to Scrap google scholar to extract meta data related to genome editing in plants. How can I achieve this?

For academic purpose, I want to Scrap publications from Google Scholar related to the topic of "genome editing in plants" to extract specific metadata from the scrapped publications, such as:...
Hamza's user avatar
  • 1
-1 votes
0 answers
13 views

How to perform backpropagation in feedforward multi-layer networks?

My given neural network is in the picture (https://i.stack.imgur.com/gg8E4.png). Each hidden neuron uses a sigmoid activation function σ(t) = 1/(1 + exp(−t)). The loss is the mean square error and the ...
Anonymous's user avatar
-3 votes
0 answers
17 views

Searching for Cancer-Related Healthcare Datasets for AI Model Training [closed]

I'm in the process of developing an Artificial Intelligence (AI) model focused on healthcare, specifically cancer diagnosis, treatment, and patient outcomes. I am seeking datasets that can provide ...
Anand 's user avatar
-1 votes
2 answers
40 views

Python as_strided method, how does it work?

I am trying to do some anomaly detection in one of my assignments and I am trying to create sliding windows, but I want to make them so that one window does not overlap with the other. For example ...
Rares Popa's user avatar
0 votes
0 answers
15 views

Why is this .csv file not woking in Weka?

I am working on a data mining project using Python sci kit and Weka. The dataset I am using is text-file and I have converted it to CSV using Python (with Pandas library). Unfortunately when I try to ...
nachoLover's user avatar
0 votes
0 answers
22 views

Decision Tree data as category

i want to draw a tree based on this data set: Source Port,Destination Port,NAT Source Port,NAT Destination Port,Bytes,Bytes Sent,Bytes Received,Packets,Elapsed Time (sec),pkts_sent,pkts_received,Class ...
shadow_9's user avatar
0 votes
0 answers
9 views

How to get K-means unsupervised learning algorithm on orange3?

I installed orange3 using miniconda. I can not find K-Means unsupervised learning algorithm. where as other unsupervised learning algorithms are there. I tried installing the educational add-on but K-...
MImi's user avatar
  • 1
2 votes
2 answers
102 views

I am trying to create an accumulator function for an vector of sets. But I keep getting strange results

Basically, I have a vector of sets. I'd like to merge the sets based on a criteria (you can see this criteria in the if statement). More than two sets may be merged. Here is an example data set: [{:...
Braden Christopher's user avatar
0 votes
1 answer
22 views

R Code Warning: number of items to replace is not a multiple of replacement length

From the Iris dataset, I have created the following confusion matrix: In attempting to perform multiclass classification using the one vs many strategy, I have come up with the following R code: mtrx ...
JTruant's user avatar
  • 399
-1 votes
1 answer
67 views

Converting Python dataframe manipulation code to R

I have a dataframe manipulation in Python that I am trying to translate into R. However, I'm running into issues with my R translation. col_names = China_2.columns provinces = col_names[0] days = ...
Zavosh Ghorbanpour's user avatar
0 votes
0 answers
15 views

How to combine non-sequence features in BiLSTM for prediction?

I'm new to LSTM. Now I have a task about destination prediction of online car-hailing, each order is the basic unit of the dataset, and the following are the features: feature_1: GPS trajectories. ...
Crixus L's user avatar
0 votes
0 answers
8 views

Comparing topological structure of similar data sets

Are there any methods within Topological Data Analysis that would allow me to check for the existence of similar topological substractures within both point clouds? Example (a bit artifficial): I have ...
josbenian's user avatar
0 votes
0 answers
18 views

Process mining dependency matrix issue

I've been playing around with various tools to try and understand / generate process mining dependency matrices. This is my smalldataset : patient handling handling_id registration_type time ...
steve's user avatar
  • 393
0 votes
1 answer
72 views

ValueError: Target size (torch.Size([555, 3])) must be the same as input size (torch.Size([2, 3, 320, 320]))

def train_loop(dataloader, model, loss_fn, optimizer, device): size = len(dataloader.dataset) model.train() for batch, (X, y) in enumerate(dataloader): # Move data to the same ...
heb gs's user avatar
  • 1
0 votes
0 answers
37 views

Request failed due to timeout issue on comtradeapicall

Here's the issue: I have code for a Comtrade call. I'm iterating for each month in the year and splitting 5 HS Codes for each API call. Here are the details of my Python code: import comtradeapicall ...
dhimasari95's user avatar
0 votes
0 answers
17 views

How to calculate incrementality of new SKU in portfolio?

I'm struggling to find a data science concept/approach that would help me answer a super simple business question: what is incremental sales from the big newness the company just launched? Context: ...
haver24's user avatar
  • 241
1 vote
0 answers
91 views

How can I get natural language data from Instagram? Should I use the official API or is it better to use data scraping methods?

I am working on a data analytics project to research the rise of toxic masculinity on Instagram amongst Persian language users. I had a straightforward method to get data from Twitter and was able to ...
Ahmad Mostafavi's user avatar
-3 votes
1 answer
37 views

How to set the equations to implement a Kalman filter for predicting the foot traffic of a restaurant?

I want to implement a Kalman filter to predict the actual daily number of dine-in customers in a restaurant with the two sets of time series data below. Daily total number of people entering the ...
starklikos's user avatar
0 votes
0 answers
60 views

Should different association rule mining algorithms return different results

I have a dataset of air quality values. For the research that I am writing, I am comparing the Apriori, ECLAT and FP-growth algorithm against eachother. For the Apriori and FP-growth algorithms I have ...
otiv33's user avatar
  • 3
0 votes
1 answer
19 views

Scrapy Problem - Whan I ran scrapy file, there is no output

` import scrapy class AudibleSpider(scrapy.Spider): name = 'audible' allowed_domains = ['www.audible.com'] def start_requests(self): yield scrapy.Request( url='...
Sazzad Hossain Nibir's user avatar
-1 votes
1 answer
16 views

BeautifulSoup methods

I had install BeautifulSoup but when I use any function in it I get the same error below AttributeError: ResultSet object has no attribute 'find'. You're probably treating a list of elements like a ...
Ahmed Elnazer's user avatar
0 votes
0 answers
24 views

Automating Support Calculation for Efficient FP-Tree Based Database Compression

I'm currently working on compressing a transactional database using the FP-tree algorithm. While adjusting the support threshold, I've encountered a trade-off: setting a very low support value results ...
spandana enamandram's user avatar
1 vote
0 answers
52 views

Compression of Database using FP trees

I have a dataset of around 200 MB, containing only integers. My goal is to use FP-trees to compress this dataset and reduce the final output file size. This is for academic purposes so I can't use ...
spandana enamandram's user avatar
0 votes
0 answers
28 views

Logging and Scraping with Selenium

I tried to Log in into website tanfonline.com . In order to scrape data with selenium library in python, I need to log in. However, the server Identified me as a robot so I tried to use ...
Ido Kobi's user avatar
0 votes
0 answers
316 views

What's the difference between Quantile Plot and Q-Q Plot?

I am trying to understand the theory behind the Q-Q Plots. So far, I can tell that Q-Q plots are used to plot one univariate quantile distribution against the corresponding quantile values of another ...
noobie's user avatar
  • 3
0 votes
0 answers
58 views

crawler break down, DUPEFILTER record 20000000+ uniqure url but jl record only 700000+ lines

I run a crawler with command "scrapy crawl any" in the dos window on my work computer with 16G ram and i5 12400 cpu. Many times it broke down after the computer run slower and slower. ...
Jobs Fan's user avatar
0 votes
0 answers
444 views

Bypassing human verification with selenium library

as part of a project, I entered site [Taylor & Francis online] ("https://www.tandfonline.com/loi/nvpp20") using selenium library in python to bypass human verification . I tried this ...
Ido Kobi's user avatar
0 votes
0 answers
10 views

Data Mining small concept:Implicit and Explicit Data

Is there any criteria for identifying if data is implicit or explicit in data mining I know data must be explicit and non-trivial for data mining but how do you identify if data is implicit or ...
Secret Coder's user avatar
0 votes
1 answer
34 views

cookies and verification problem in scraping

After running this code as part of a project, in order to enter login screen : import pandas as pd import requests from bs4 import BeautifulSoup from selenium import webdriver from selenium.webdriver....
Ido Kobi's user avatar
0 votes
2 answers
55 views

Clicking a link with selenium library

I have a project scraping data from tandfonline which I have institutional access to. To get the data from each article I need to click the issue button each time with selenium library, which has a ...
Ido Kobi's user avatar
0 votes
0 answers
53 views

Count repeating intervals in a plot [duplicate]

In the image, you can see five repetitive peak sections. I want to write an algorithm that counts this number, but I am not sure how. I tried to draw a regression curve using seaborn.lmplot, but it ...
helpme's user avatar
  • 19
0 votes
0 answers
8 views

how can I the get data of the posting users in Instagram from Facebook GraphAPI?

Facebook in the GraphApi send post details without user information. but I need to users data for my analytics production. in GraphApi docs write you cant get posting users media object.GraphApi ...
mohamad hafezi's user avatar
0 votes
0 answers
34 views

Perform K-means Clustering in RapidMiner - Dealing with Categorical Variables

I have a table customer_segmentation where each entry corresponds to a unique customer, and the structure of the table is as follows: customer_id: (integer) Unique key to identify each customer gender:...
Tiago's user avatar
  • 41
0 votes
1 answer
131 views

Problems extracting text from pdfs in PYTHON

I'm doing a text extraction from some pdfs and return in the terminal what is the extracted text using python. However, there are some pdfs that are returning me instead of the text, special ...
EKatia's user avatar
  • 1
0 votes
0 answers
38 views

How to standardize data with different dilution factors and minimum values for each dilution factor?

I have been having trouble running statistical analysis on a dataset from a mouse experiment. The dataset is derived from 3 different dilution factors (1:10, 1:20, 1:40) with each having different ...
Sid Lingatlu's user avatar
0 votes
0 answers
28 views

why classification trees doesn't give a single split?

I want to come up with a decision tree with a single split. But with the following code, the node is not split. dt<-rpart(data$y ~ . , data = data, control = rpart.control(minsplit=1,...
san's user avatar
  • 11
0 votes
0 answers
23 views

how to fix id key not found in pentaho spoon

im trying to create a fact table from my dimension tables using foreign keys and primary keys but when im trying to execute it gives me that id key couldnt be found id key coudlnt be found but i m ...
Belbox's user avatar
  • 1
0 votes
1 answer
262 views

Error when running tensorflow install package

I'm trying to install tensorflow and i don't know how to fix this error: Installing collected packages: tensorflow-intel, tensorflow ERROR: Could not install packages due to an OSError: [Errno 2] No ...
rodrigoogalde's user avatar
0 votes
0 answers
15 views

Thresholding Neural Network outputs

So I currently have this graph dataset, which have two classes, and I intend to do a binary classificaiton problem. The following is the data’s information from torch_geometric.datasets import ...
Hai Nguyen's user avatar
0 votes
0 answers
223 views

Installing SQL Server 2019 Analysis Services on a SQL Server 2022

I noticed that in SQL Server 2022, SSAS' Data Mining is discontinued as per this page: https://learn.microsoft.com/en-us/analysis-services/data-mining/data-mining-wizard-analysis-services-data-mining?...
Lewis Hellewell's user avatar
0 votes
0 answers
99 views

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs

`here when i run the code , i face with the type error like : TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according ...
sanaullah sediqe's user avatar
0 votes
0 answers
17 views

How to do association rule mining with different characteristics of one variable?

I am working on a task where I have data about patients with the variables reportid (date of treatment), type (T for Treatment, S for Symptoms, C for conditions) and a variable names with the assigned ...
Marlene's user avatar
0 votes
0 answers
28 views

How to select weeks with no precipitation recorded in R? [duplicate]

I conducted several experiments that lasted for approximately one week each. Precipitation data was recorded at 15-minute intervals, and my data looks like this I want to select only those weeks in ...
Ahsk's user avatar
  • 267
0 votes
0 answers
52 views

Association Rules with Orange3 using table of orders/bills - How to convert into: cloumns is Items_name, Row is OrderID for associate?

I have a table of orders where we have many of orders, each order is an item. And in excel, you have 2 columns: OrderID and ItemName. So if u have 4 items in one order, you will have 4 row have the ...
Kolnh Jiu's user avatar
1 vote
1 answer
62 views

LSTM Model not improve on sentiment analysis, what im doing wrong?

I have sentiment data that contains 3 labels (positive, negative, neutral) and i have 3233 row data, already tested on naive bayes and svm model, my data got 90 % accuracy on naive bayes, and 92 % ...
Albert Deo HK's user avatar
1 vote
1 answer
42 views

How to select a certain period based on a condition in R?

I have conducted several experiments that lasted for about a week. For each week, I'd like keep periods/rows when precipitation began to the end of the week. I know how to discard dry weeks, but I'm ...
Ahsk's user avatar
  • 267
0 votes
0 answers
24 views

Orange multiple choice survey import

is there any way to import answers to multiple choice questions of a survey other way than binary? The answers are purely categorical. Right now the chosen answers of multiple choice question are in ...
Tim's user avatar
  • 1
-1 votes
1 answer
30 views

Oracle / SQL - summary table with grouped data

I have following raw data table: Account Owner Strategy POSITION_FLAG 50001 CompanyA Equity 1 50002 CompanyA Equity 0 50002 CompanyB Fixed Income 1 Now I want to produce summary table: X Equity ...
Radek's user avatar
  • 15
0 votes
0 answers
19 views

In an imbalance classification problem, is it fair to balance the test data by removing some negative examples?

I'm working on an imblance classification problem, regarding the evaluation on the test set I had the following questions Is it a fair practice to use balanced test set for evaluation ? To create ...
suresh kumar A's user avatar
0 votes
0 answers
15 views

Is PCA required for dimensionality reduction in data mining tasks?

I wonder if it is really necessary to use PCA to reduce the dimensionality of input variables in actual data mining tasks. For interpretability reasons, shouldn't adding regularization terms be a ...
ZeroLoveSeA's user avatar

1
2 3 4 5
62