Questions tagged [vectorization]

Vectorization refers to a programming paradigm where functions operate on whole arrays in one go. This affords benefits in terms of function calls, memory access, parallelization and code expressiveness. Some programming languages, such as MATLAB, are optimised to give the best performance when vectorized.

vectorization
Filter by
Sorted by
Tagged with
2 votes
1 answer
615 views

Pandas: vectorization of Conditional Cumulative Sum

I'm trying to vectorize a for loop in pandas to improve performance. I have a dataset comprising of users, products, the date of each service as well as the number of days supplied. Given the ...
Alex C.'s user avatar
  • 53
2 votes
2 answers
662 views

How to calculate the mean of n consecutive columns?

I have a dataframe like this: import pandas as pd df = pd.DataFrame({'A_1': [1, 2], 'A_2': [3, 4], 'A_3': [5, 6], 'A_4': [7, 8], 'B_1': [0, 2], 'B_2': [4, 4], 'B_3': [9, 6], 'B_4':...
Cleb's user avatar
  • 25.6k
4 votes
2 answers
2k views

Multiply two matrix by columns with python

I have two matrix: A = [a11 a12 a21 a22] B = [b11 b12 b21 b22] And I want to multiply all its columns (without loops) in order to obtain the matrix: C =[a11*b11 a11*b12 a12*b11 ...
DiegoDZ's user avatar
  • 189
2 votes
1 answer
86 views

mean till 25 percentile for rows of an array containing nan

I have a 2d array x, with each row having a different number of nan values: array([[ nan, -0.355, -0.036, ..., nan, nan], [ nan, -0.341, -0.047, ..., nan, 0.654], [ .016, ...
dayum's user avatar
  • 1,083
2 votes
3 answers
165 views

How to select value from array that is closest to value in array using vectorization?

I have an array of values that I want to replace with from an array of choices based on which choice is linearly closest. The catch is the size of the choices is defined at runtime. import numpy as ...
dranobob's user avatar
  • 806
1 vote
1 answer
237 views

numpy vectorized assignment of sequences [duplicate]

Is there a vectorized assigment of elements to sequences in numpy like in this discussion? for instance: xx = np.array([1,2], dtype=object) expanded = np.arange(xx, xx+2) instead of loops: xx = ...
ecoe's user avatar
  • 5,154
2 votes
1 answer
178 views

Loop vectorization gives different answer

I am building some unit tests and find that my code gives a slightly different result when vectorized. In my example case below, an array a is summed in one dimension and added to an initial value x. ...
Ross's user avatar
  • 2,149
0 votes
1 answer
43 views

How can I vectorize this piece of code for loading cross-validation from file?

So I am supposed to load the cross-validation folds from a file (10 iterations, each is 10-folds, so a total of 100, stored sequentially in a 1-dimensional array). Here is the file: https://www....
Dang Manh Truong's user avatar
3 votes
1 answer
423 views

Creating sliding windows of NaN padded elements off 1D NumPy array

I have a time series x[0], x[1], ... x[n-1], stored as a 1 dimensional numpy array. I would like to convert it to the following matrix: NaN, ... , NaN , x[0] NaN, ... , x[0], x[1] . ...
motam79's user avatar
  • 3,714
1 vote
2 answers
959 views

Python: using np.apply_along_axis with a function returning array

I'm trying to do something like the following import numpy as np arr = np.array([0,1,2,3]) fn = lambda x: np.array([x/2,x*2]) res = np.apply_along_axis(fn,0,arr) That is, I want to use np....
user avatar
6 votes
3 answers
1k views

cumulative argmax of a numpy array

Consider the array a np.random.seed([3,1415]) a = np.random.randint(0, 10, (10, 2)) a array([[0, 2], [7, 3], [8, 7], [0, 6], [8, 6], [0, 2], [0, 4], [...
piRSquared's user avatar
  • 291k
1 vote
1 answer
87 views

Use omp simd without a for

I have the following instructions: unsigned long int xDiff = seq1.x ^ seq2.x; unsigned long int yDiff = seq1.y ^ seq2.y; unsigned long int zDiff = seq1.z ^ seq2.z; It's possible to vectorize this ...
Guilherme Torres Castro's user avatar
0 votes
1 answer
85 views

How to convert "for loop" to "apply family functions"

My data frame looks like this. (it has ~ 300,000 rows) I want to do this operation faster and elegantly. How can i overcome this problem. Thanks in advance. df <- data.frame( X = sample(0:1, ...
xfactor's user avatar
  • 17
1 vote
2 answers
65 views

Equivalent using numpy

For some work that I am doing recently, I need the following operation to be done. def myfunc(a, b): return a*b # some operation here a = [1,2,3] b = [2,4,6,8] print [[myfunc(i, j) for i in a] ...
Ramesh-X's user avatar
  • 4,933
1 vote
1 answer
401 views

GCC no vectorization output?

I am trying to vectorize a loop using ivdep but I'm not getting any output. I looked at the documentation and used the flags mentioned there. This is my code: int main() { int a[100], b[100]; int ...
Moody's user avatar
  • 1,347
8 votes
3 answers
724 views

OpenMP paralelization inhibits vectorization

I am new to OpenMP and I am trying to paralelize following code using OpenMP: #pragma omp parallel for for(int k=0;k<m;k++) { for(int j=n-1;j>=0;j--) { outX[k+j*m] = inB2[j+n * k] /...
Honza Dejdar's user avatar
1 vote
2 answers
126 views

Vectorized numpy 1-d reclassification

I have a large numpy 1-d containing about 700,000 classes. In addition, I have another similar sized array which contains the new values of the classes. Example arrays original_classes = np.array([...
Wilmar van Ommeren's user avatar
10 votes
4 answers
8k views

Vectorized NumPy linspace for multiple start and stop values

I need to create a 2D array where each row may start and end with a different number. Assume that first and last element of each row is given and all other elements are just interpolated according to ...
dayum's user avatar
  • 1,083
0 votes
1 answer
453 views

numpy. Sum multidimensional array at specific indexes

I'm trying to replace something like this code, for a vectorized efficient operation using numpy. counter = 0 idxs = [1, 3] lists = [[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]] for l in lists: for ...
FranGoitia's user avatar
  • 1,985
1 vote
1 answer
1k views

Where can I find an AVX exponential double-precision function?

I was recently researching implementations of vectorized exponential functions. It's actually very simple to find code for single-precision, but very hard to find code for double-precision. I haven't ...
Baptiste Wicht's user avatar
3 votes
2 answers
369 views

Sum of outer product of corresponding lists in two arrays - NumPy

I am trying to find the numpy matrix operations to get the same result as in the following for loop code. I believe it will be much faster but I am missing some python skills to do it. It works line ...
Daerken's user avatar
  • 33
3 votes
1 answer
66 views

Vectorization of removal of non-periodic offset from data, where offset values are stored in same vector as data

I have two column matrices with an equal number of rows: One of the column matrix (let us call it T) is a non-periodic set of unique integers sorted in increasing order. T can be simulated by: ...
Ugo Zanforlin's user avatar
0 votes
1 answer
43 views

cellarray of matrix multiplication vectorization

I have A and B a cell array of matrices inside. I want to obtain C A = [18x18 double] [18x18 double] [18x18 double] [18x18 double] [18x18 double] [18x18 double] [18x18 ...
Ka Wa Yip's user avatar
  • 2,704
6 votes
3 answers
3k views

Vectorized searchsorted numpy

Assume that I have two arrays A and B, where both A and B are m x n. My goal is now, for each row of A and B, to find where I should insert the elements of row i of A in the corresponding row of B. ...
Tingiskhan's user avatar
  • 1,325
10 votes
2 answers
13k views

Computing Jaccard Similarity in Python

I have 20,000 documents that I want to compute the true Jaccard similarity for, so that I can later check how accurately MinWise hashing approximates it. Each document is represented as a column in ...
Magic8ball's user avatar
2 votes
2 answers
1k views

Normalizing a numpy array

Given an array, I want to normalize it such that each row sums to 1. I currently have the following code: import numpy w = numpy.array([[0, 1, 0, 1, 0, 0], [1, 0, 0, 0, 0, 1], ...
wwl's user avatar
  • 2,055
1 vote
1 answer
263 views

How to find mean in kmeans in single shot using numpy

I have a function: def update(points, closest, centroids): return np.array([points[closest==k].mean(axis=0) for k in range(centroids.shape[0])]) It basically the update of centroids step in ...
frazman's user avatar
  • 32.6k
11 votes
1 answer
1k views

Condition for memory access conflict in memory-banked vector processors

The Hennessy-Patterson book on Computer Architecture (Quantitative Approach 5ed) says that in a vector architecture with multiple memory banks, a bank conflict can happen if the following condition is ...
Parth Thakkar's user avatar
1 vote
2 answers
479 views

How to evaluate function for input array [matlab]

Suppose I have a function func.m in matlab function [sum] = func(x) d = length(x); sum = 0; for i = 1:d sum = sum + x(i)^2; end end and two vectors defined as ...
user avatar
-2 votes
3 answers
1k views

Sum without use of loop in R

I've a vector. basket <- c(4,5,10,102,10); If I've to sum, I could call sum(basket) If I've to use for loop to find out sum, total <- 0; len <- length(basket); for(i in 1:len) {total &...
tired and bored dev's user avatar
1 vote
2 answers
907 views

Python Pandas iterrows and set_value as vectorized function

I am using this function in one of my programs, and from what I have researched better performance can be achieved through vectorization. How can I achieve the same resolution using a vectorized ...
Aran Freel's user avatar
  • 3,155
1 vote
2 answers
874 views

efficiently fill a tensor in numpy

I have a numpy array X of size (n, m) and of type np.uint8 (so it only contains values in [0, 255]). I also have a mapping f from [0, 255] to [0, 3]. I want to create an array Y of shape (4, n, m) ...
dhokas's user avatar
  • 1,771
2 votes
1 answer
1k views

Iterating over numpy array without for loop

I am using numpy arrays in my codes very frequently. Its speed and convenient indexing rules are very useful. Now, I am searching how to avoid 'for' loops in order to make execution time faster. For ...
Elgin Cahangirov's user avatar
1 vote
2 answers
145 views

Creating a dataframe by concatenating substrings of non-uniform lengths

The raw data (.txt) file has 65926 elements, each containing a 142 character string. Here's the dropbox link to the raw data file. The task is to divide that 142 character string into 37 smaller ...
android16.5's user avatar
1 vote
1 answer
595 views

python creating a 2d array/dataframe with variable row elements

I have a timneseries over a number of days where for each day, I have a variable number of datapoints. A sample dataframe is generated bwlow: n=10,20 init=datetime.datetime(2016, 7, 24, 0, 0) df=pd....
dayum's user avatar
  • 1,083
5 votes
3 answers
2k views

How to dynamically reshape matrix block-wise? [duplicate]

Let's say I have A = [1:8; 11:18; 21:28; 31:38; 41:48] Now I would like to move everything from column 4 onward to the row position. How do I achieve this? A = 1 2 3 4 5 6 ...
Gregor Isack's user avatar
  • 1,111
2 votes
1 answer
423 views

Point inside polygonal region with polygonal hole

I am using matplotlib.path.Path to check whether a set of points are inside a region bounded by polygons (polygonal region with a polygonal hole). My approach involves two checks and a loop: import ...
Numaerius's user avatar
  • 383
0 votes
0 answers
58 views

python pandas - would like to vectorize this loop function and apply to a panel

Would like to lean more on numpy/pandas vectorization than run the following loop procedure over a huge dataset. import pandas as pd list1 = [[21,101],[22,110],[25,113],[24,112],[21,109],[26,108],[...
MJS's user avatar
  • 1,603
0 votes
1 answer
567 views

Is there an equivalent of _mm_slli_si128(__m128i a, int num) for floats?

Let's say I have a vector of 4 floats: __m128 vector = |f0|f1|f2|f3| (pseudocode) My intention is to transform that variable into this: |0.0|f0|f1|f2| Doing a shift right would appear to be the ...
ismarlowe's user avatar
  • 137
1 vote
1 answer
143 views

get max value of multiplication of column combinations and their respective index in python

I have a numpy array of M*N dimensions in which each element of the array is a float with a value between 0-1. Input: for simplicity purpose lets consider a 3*4 array: a=np.array([ [0.1, 0.2, 0.3, 0....
darekarsam's user avatar
0 votes
2 answers
43 views

Copying array contents to cell in vectorised way MATLAB

I have a nx3 array A. I want to create a cell B such that: for i=1:size(A,1) B{i}=A(i,:); end Can i do this in a vectorised way without for loop or any faster than this in Matlab? EDIT: This ...
user_1_1_1's user avatar
15 votes
2 answers
2k views

How does gensim calculate doc2vec paragraph vectors

i am going thorugh this paper http://cs.stanford.edu/~quocle/paragraph_vector.pdf and it states that " Theparagraph vector and word vectors are averaged or concatenated to predict the next word ...
jxn's user avatar
  • 7,883
18 votes
2 answers
2k views

How to write a cumulative calculation in data.table

A sequential, cumulative calculation I need to make a time-series calculation, where the value calculated in each row depends on the result calculated in the previous row. I am hoping to use the ...
Douglas Clark's user avatar
4 votes
1 answer
1k views

In a 3d array in numpy. How can I extract indices of max element in third dimension?

Example input 3D array of shape (2,2,2): [[[ 1, 2], [ 4, 3]], [[ 5, 6], [ 8, 7]]] My 3d array has a shape of (N, N, N), in above example N = 2. I need to get all indices such that index for ...
bits's user avatar
  • 1,645
1 vote
1 answer
46 views

Build up a cell array with different amount of zeros in Matlab

I want a cell array with 1 zero in the first column, 2 zeros in the next etc up to n. For this Im trying to use arrayfun the following way: index = 1:n; a{index}=arrayfun(@(index) zeros(index,1),1); ...
user1661303's user avatar
2 votes
1 answer
103 views

Fast sequential addition to rectangular subarray in numpy

I have come across a problem which is to rewrite a piece of code in vectorized form. The code shown below is a simplified illustration of initial problem K = 20 h, w = 15, 20 H, W = 1000-h, 2000-w ...
xolodec's user avatar
  • 843
7 votes
3 answers
3k views

List of dictionaries from numpy array without for loop

Is there a way to vectorize an operation that takes several numpy arrays and puts them into a list of dictionaries? Here's a simplified example. The real scenario might involve more arrays and more ...
durden2.0's user avatar
  • 9,362
10 votes
1 answer
7k views

Vectorizing NumPy covariance for 3D array

I have a 3D numpy array of shape (t, n1, n2): x = np.random.rand(10, 2, 4) I need to calculate another 3D array y which is of shape (t, n1, n1) such that: y[0] = np.cov(x[0,:,:]) ...and so on for ...
dayum's user avatar
  • 1,083
2 votes
1 answer
50 views

Using anonymous function rather than looping for summing up values in cell array

I have a matrix called data. It contains 3 columns, company name, company value & company currency, bit like below. Name Value Currency ABC 10 USD MNO 5 ...
mHelpMe's user avatar
  • 6,498
0 votes
0 answers
82 views

Calculate peaks of rows in matrix vectorized in matlab

I've got a series of row vectors in a matrix. I want to calculate the peaks of each row and assign them to a new array. Ex: A = [1,3,4,2,1,3,1;5,6,7,6,7,6,5] [peaks, inds] = myfunc(A) peaks = 4 ...
user1661303's user avatar

1
80 81
82
83 84
135