Questions tagged [resampling]
Resampling is any of a variety of methods for estimating the precision of sample statistics by jackknifing or bootstrapping. In it also used to validating models by using random subsets (bootstrapping, cross validation).
986
questions
0
votes
1
answer
44
views
Interpolating time-series with the date format
I have the following time series data:
Image
I want to interpolate the data monthly or yearly, and my code is as follows:
df = pd.read_csv('Data/data_processing.csv', encoding='latin1')
df['Date'] = ...
0
votes
0
answers
59
views
Why does sampling without replacement in my function just duplicate my output?
I've written two functions, one works, one doesn't. Was wondering why sampling in the first example just duplicates my results - why is it not creating new shufflings without replacement for each ...
0
votes
0
answers
71
views
gdalwarp setting xres and yres
I'm using gdalwarp for resampling a GeoTiff from its original resolution (5m, 5m) to a target resolution (10m, 10m):
gdalwarp -tr 10 -10 input.tif output.tif
I'm using a negative yres because when I ...
0
votes
1
answer
68
views
Optimize metrics for Fraud Detection Imbalanced Data
I would need your help to improve my model performance. As mostly happens for fraud detection, I have an imbalanced dataset (0.1/0.9). I would like to optimize the recall for my target 1 and 0, ...
0
votes
0
answers
38
views
Hypothesis testing - Newbie blockers
Brief : I'm from manufacturing industry, a processing machine in our production line used to do pressing, polishing and QA in one line. Now we have a new machine that will perform these separately at ...
0
votes
2
answers
53
views
Upsample timeseries in pandas DataFrame and interpolate
I have an example DataFrame below where my sampling frequency = 250 Hz (1 sample every 4 ms):
df =
datetime sample_value
2000-01-01 00:00:00.004 56
2000-01-01 00:00:00.008 ...
0
votes
0
answers
29
views
MATLAB: Same-length, same-type vectors take up different amounts of space as .mat files
I have neural recordings of two resolutions, one sampled at 2048Hz and the other at 32768Hz. I implemented an analysis pipeline for the 2048Hz recording that I want to apply to the other, higher-...
1
vote
2
answers
22
views
Resampling Partial Month to Full End of Month - Pandas
Putting this out there to see if there is a feasible solution for this.
Suppose I have a volumes for every month that say looked liked :
But if the start date for these volumes to be assigned is ...
0
votes
0
answers
44
views
Does sklearn pipeline cross-validation resample the training fold data only and remain the hold-out/validation fold unbalanced at each CV fold?
For an imbalanced dataset, at each K-fold of cross-validation, I want to do SMOTE resampling for the training fold data to make it balanced, and remain the hold-out/validation fold data unbalanced. ...
0
votes
0
answers
30
views
Downsampling raster (average) with regard to minimum filled pixel
I want to downsample raster by scale of 3 (from 5m to 15m cellsize). I used rasterio with resample average method. However this generalize area with plenty of NA pixels and created undesirable result. ...
0
votes
1
answer
80
views
0.632+ Bootstrap Prediction Intervals in R from a caret-trained model
I'm trying to write a function in R that calculates a central prediction and upper and lower prediction intervals from a trained caret model (i.e., a "train" object) using the 0.632+ ...
2
votes
2
answers
61
views
How to perform nested resampling in R - Diversity index uncertainty
I'm currently trying to apply the resampling method described in Herrmann et al. (2022) with an outer and an inner resampling loop :
My dataset is composed of 100 rows by 60 columns. Rows correspond ...
1
vote
2
answers
37
views
Pandas timestamp ceil resampling
I need to resample a timestamp field, but I'm not getting success with pandas' resample method. The table below presents resampling using two approaches: resample and groupby with ceil. Is there a way ...
0
votes
0
answers
32
views
Resample hourly timeseries to MS granularity with offset
This question can be seen as an extension of the following question:
Resample hourly TimeSeries with certain starting hour
I have the following dataframe:
import pandas as pd
index = pd.date_range(...
0
votes
0
answers
27
views
mutate resampling (rset) object
I have created resampling folds using vfold_cv(). I need to adjust the $data dataframes within the $splits in each fold. After doing this, however, the object is no longer a rset object. How can I ...
1
vote
1
answer
31
views
Optimizing multiple pandas resample operations
I have this dataset that consists of type of action, x, y, and z coordinates, label, and timestamps. What I want to do is to calculate some variables based on the coordinates and thus get more ...
0
votes
0
answers
25
views
NetCDF Data Extraction and Interpolation for a Specific Region
I have a global NetCDF file form ISIMIP. I need a interpolate for a specific region. To do that I used a mask with the following code for a specific region. But the result I get is not exactly what I ...
0
votes
1
answer
52
views
Resampling a number of rows 1000 times and calculate de variance
I want to select 3 random rows per group for one especific variable. I want to run 1000 times. Then I want to calculate the mean, the variance and the mean's confidence intervals of these 1000 ...
0
votes
0
answers
5
views
Reiteration of an analysis on resampled subgroups
I am new to this forum, so I will be very grateful for your help.
I have dataset with nested observations (2 individuals within 2922 pairs). What I need to do is to make a subgroups of the dataset ...
3
votes
2
answers
104
views
How to efficiently sample with replacement at two levels of hierarchy with data.table/dplyr?
I'm trying to use data.table/tidyverse to efficiently sample at two levels:
Level 1 is Hospital ID ( hospital_id )
Level 2 is Doctor ID ( doctor_id )
I need to first sample with replacement $N$ ...
1
vote
1
answer
51
views
Images sizes not matched when downsampling then upsampling images
I am trying to downsample an image (for speed), run prediction, then upsample it back. Due to rounding, I get mismatches with the original image size for some pixel dimensions/voxel sizes. What is the ...
0
votes
0
answers
35
views
Stratified Sampling in Python: wrong final sample size?
First time posting here.
I am using this line to get a stratified sample of my DataSet, using the variable TamañoMuestral = 0.05
DataFrameMAE = DatosPoblacionales.groupby(NombreDeEstrato, group_keys=...
0
votes
0
answers
31
views
How to downsample a PCM stream in android with kotlin
I am attempting to show the audio intensity in Android and in order to do it, I am recording data in stereo PCM 16 bit with AudioRecord, so I have a buffer with the values.
The recording sample rate ...
0
votes
0
answers
31
views
Imbalanced target (over and under sampling)
I'm working on a machine-learning project and my target is quite imbalanced (only 9% positive), so I need to resample the data to ensure that the model doesn't get biased toward the majority class, ...
1
vote
2
answers
39
views
Boostrap of Dataframe for Re-estimation
I have a given dataset and want to create a bootstrap (with resampling) of 10 replicates of the dataframe and have new ids assigned to them in the final dataset. Please guide me on this.
Random.seed!(...
0
votes
1
answer
49
views
resampling/"upscaling" using scipy.interpolate.RectBivariateSpline no diffrence
I am testing out scipy.interpolate.RectBivariateSpline for a project where I want to upscale some data to achieve better resolution. My attempt at using both scipy.interpolate.RectBivariateSpline and ...
0
votes
0
answers
34
views
Resample data with given sample (Python, numpy)
I consider two data D1 = (x1, y1) and D2 = (x2, y2), where x1, x2, y1 and y2 are arrays and thus D1 and D2 each describe a graph. Problem: x1 and x2 are different, making it difficult to compare the ...
0
votes
0
answers
22
views
How can I bootstrap a variable from selected subgroups in R?
In R I have a data set that covers the southern Baltic Sea. Some stations were sampled once a year with only one grab sample. Other stations were sampled only once with only one grab sample and others ...
1
vote
1
answer
251
views
Feature Selection Using Bootstrap Resampling, LASSO and Stepwise Regression
In this paper, the authors perform radiomics feature selection for survival prediction by:
Bootstrap resampling the dataset x 1000
Fitting cross-validated LASSO models to each the resampled data sets
...
2
votes
1
answer
381
views
Pandas Datathinning: How can I effectively reduce the number of points in a Series while still preserving the original overall trend?
I have some time series data and a parameter of interest in a dataframe - there are hundreds of thousands of rows, but for a large portion of them there is only an extremely small change (if any at ...
0
votes
1
answer
395
views
How to Resample GeoTIFF on Python?
I am trying to resample a GeoTIFF file from pixels representing 30m² to pixels representing 1000m² on Python.
I have a geotiff that has the data according to those coordinates. The spatial resolution ...
0
votes
2
answers
42
views
Resample daily data to hourly dataframe with dateindex
I have this dataframe:
date,Prices
2018-01-01, 45.338
2018-01-02, 39.955
I would like to resample this hourly taking advantage of dateindex. In particular I would like to assign the same value to ...
0
votes
1
answer
149
views
How to set sample rate in a VST3 plugin
I try to create a vst3 plugin that can resample audio data to a given sample rate. Just like the "Resampling Plugin" of Wavelab.
How can a vst3 plugin change the output sample rate?
A ...
0
votes
0
answers
69
views
Output MRI images have value 0 (almost) everywhere after resampling using SimpleITK
I have 3-dimensional NIfTI MRI images that I would like to resample to uniform voxel size (1, 1, 1). This works fine:
img = sitk.ReadImage(img_path)
output_spacing = [1.0, 1.0, 1.0]
input_spacing = ...
1
vote
1
answer
188
views
Bootstrapping on cluster level in python
I need to bootstrap a statistic on cluster level in python.
I have multiple measurements per cluster and I want to perform a bootstrapped statistic (such as R-squared) based on sampling with ...
0
votes
1
answer
125
views
How can I apply preprocessing in each cross-validation fold trained on each train part of the fold using tidymodels?
I try to use the tidymodels R-package for an ml pipeline. I can define a preprocessing pipeline (a recipe) on the training data and apply it to each re-sample of my cross-validation. But this uses the ...
0
votes
1
answer
155
views
Downsampling and aliasing effect in python st.resample
Does downsampling using stream.resample () in Python introduce aliasing effect?
I want to investigate a signal below 4 Hz. It has the original sampling rate of 100 Hz but I need to downsample it as ...
0
votes
0
answers
30
views
What is the difference between block bootstrapping and group k-fold cross-validation?
I know that block bootstrapping is a technique used to resample time series data as it can preserve time dependencies within the data. In particular, if the block size is one month, then data within ...
3
votes
1
answer
145
views
Pandas dataframe: resampling time intervals and dividing values proportionally?
I have the following pandas dataframe:
index
start_time
end_time
amount
foo
2023-03-11 09:45:27
2023-03-11 09:58:39
48
bar
2023-03-11 09:59:00
2023-03-11 010:09:00
20
I'm hoping to split the data ...
0
votes
0
answers
92
views
Is there any way to turn a time series from into 15 min intervals?
I have an excel with data that was recorded in random intervals for a whole day. I'm trying to turn that time series into 15-minute intervals to match another data set.
Excel File
import pandas as pd
...
-1
votes
1
answer
104
views
pandas resample with global minimum and maximum specifying the filling method
I want to resample a dataset, the minimum date of each series should be the same for each series, therefore, the minimum date of each series should be the minimum value for the date column. Same for ...
1
vote
1
answer
290
views
open cv cv2.imread a grayscale image pixel values
I am attempting to load a grayscale image using cv2.imread for further blob extraction. To start I have an image (as seen below) for which black pixels have values (typically between 0 and 1) and ...
1
vote
1
answer
184
views
Is bilinear resampling to a coarser resolution is similar to weighted spatial averaging?
I am using the xESMF python package to resample NDVI (greeness) data from 500 * 500 m to 1 * 1 degree.To clarify, I'm increasing the scale of the data. The package offers several techniques, including ...
0
votes
0
answers
145
views
How to deal with unevenly spaced time series in python (Pandas)
Good afternoon together,
I want to analyze time series data in python, but I cannot be sure in which interval the data is coming. I would guess it comes every five seconds, but maybe sometimes quicker ...
0
votes
1
answer
72
views
Missing value handling with imputation in a nested resampling procedure such that there is no information bleed from train to test
I am looking through the documentation for the nested resampling procedure in the mlr3tuning package and I do not see any way that the package can handle NA values such that any information bleed ...
0
votes
0
answers
44
views
Resample basemap in openlayers
I have an application developed based on openlayers and I load a esri basemap (https://server.arcgisonline.com/arcgis/rest/services/Ocean/World_Ocean_Base/MapServer/tile/{z}/{y}/{x})
From zoom level ...
0
votes
0
answers
81
views
Bootstrap resampling for stacked/ensemble leaner with mlr3 in R
So I'm trying to generate bootstrapped resamples for an ensemble model which throws an error. This seems to result from the duplication of row_ids; I suppose these duplicate rows should be expected ...
0
votes
0
answers
55
views
Resampling the decomposed series of trend, residual and seasonality of the data from monthly to daily
I am performing a short-time series decomposition on a dataset to evaluate trend, residual and seasonality of the data. I want to resample these trend, seasonality and residual from monthly to daily ...
0
votes
1
answer
38
views
How to perform bootstrap on weighted correlation in R?
I'm trying to bootstrap a weighted correlation in R, with the different data streams having separate weights.
Some example data:
DF = data.frame(
x = c(-0.3, 0.3, -0.18, 0.02, 0.07, 0.11, 0.20, 0.8, ...
0
votes
1
answer
81
views
Creating balanced bootstrap resamples in caret
I'm using caret to compare models for a classification problem with nested CV. Vfold in the outer loop and bootstrap (500 replicates) in the inner loop. I get this error after training knn:
Warning: ...