Questions tagged [sparse-matrix]
A sparse matrix is a data structure in which not every entry is explicitly represented. Related are sparse matrix algorithms and data structures, along with questions about implementation and analyses.
3,786
questions
0
votes
0answers
24 views
Is there any reimplementation of Matlab eigs(A, B, k, sigma) in C/Cpp? [closed]
I need to compute the generalized eigenvalue and eigenvector of two large sparse matrices A and B in C++, and it seems hard to find any pratical solution. https://github.com/yixuan/spectra doesn't ...
2
votes
1answer
30 views
How to create a sparse DataFrame from a list of dicts
I create DataFrame from a list of dicts like this:
pd.DataFrame([{"id":"a","v0":3,"v2":"foo"},
{"id":"b","v1&...
1
vote
1answer
28 views
How to ignore implicit zeros using scipy.sparse.csr_matrix.minimum?
I have two matrices mat1 and mat2 that are sparse (most entries are zero) and I'm not interested in the zero-valued entries: I look at the matrices from a graph-theoretical perspective where a zero ...
1
vote
1answer
43 views
How can create a sparse matrix out of outer function in R?
In my code, the outer function outputs a matrix. Is it possible to make that output sparse matrix? I could convert this output matrix to a sparse matrix, but I want to do something like outer(......, ...
1
vote
3answers
68 views
R: Changing diagonal of sparse matrix is very slow
I hava a sparse matrix with zeros on the main diagonal that I want to change to ones, but compared to a QR-decomposition the operation is very very slow:
mat <- matrix(c(0,1,1,1,0,1,1,1,0),ncol=3)
...
0
votes
0answers
21 views
Stack sparse matrices along new axis with scipy
I have derived d sparse matrices m[d] of size (n, n) each, and I would like to stack them along a new axis in order to build a sparse matrix of size (n, n, d).
I tried building this stacked matrix ...
1
vote
1answer
13 views
How to multiply a sparse matrix by a sparse matrix element-wise in pytorch
In pytorch, I can achieve two sparse matrixes multiplication by first turning them into a dense form
adjdense = torch.sparse.FloatTensor(indextmp, valuetmp, torch.Size([num_nodes,num_nodes])).to_dense(...
-4
votes
1answer
70 views
Within a sparse-array data-structure how does one retrieve the real position count of such a structure's item?
I have a sparse array of array entries/items as follows ...
var indexed = new Array();
indexed["5"] = [35, 2];
indexed["3"] = [20, 2];
indexed["4"] = [25, 3];
I want to ...
0
votes
0answers
33 views
Performance improvement over numpy.linalg.solve for systems of 50k x 50k
Currently I am using (I think serially unless the code is parallelized/optimized for shared memory) on multi-core AMD processor machines:
try:
B = np.linalg.solve(F1, b)
except np.linalg....
1
vote
1answer
41 views
Eigen C++: mapping a raw buffer for a Sparse Matrix
I have some troubles with mapping a raw buffer for a SparseMatrix.
1 - No problem for the Non Sparse Matrix Case with the following code:
Eigen::MatrixXd DIFFXX;
new (&DIFFXX) Eigen::Map<Eigen::...
0
votes
0answers
20 views
How can I identify the rows of a matrix that originated in a dataframe?
Within my dataframe I have a variable containing text (type: panda series). This specific variable "text" is turned into a sparse matrix (=tfidf_matrix) by way of sklearn TfidfVectorizer(). ...
-1
votes
1answer
28 views
Removing rows from a sparse matrix of class “dgCMatrix” in R
I am a bioinformatics student who is fairly new to R. I am working with a sparse matrix of class "dgCMatrix" and I'd like to find a way to conditionally remove rows in that matrix.
Click ...
0
votes
1answer
23 views
How does CountVectorizer deal with new words in test data?
I understand how CountVectorizer works in general. It takes word tokens and creates a sparse count matrix of documents (rows) and token counts (columns), that we can use for ML modeling.
However, how ...
1
vote
3answers
38 views
How to correctly match pandas multiindex dataframe multiplication for sparse data
I've searched before posting this, I've found among others this previous stack overflow post and I don't think it answers my question.
I have sparse data that I want to multiply together correctly ...
0
votes
0answers
9 views
Add same motive to all columns in a sprase matrix (in R)
I would like to add the same motive to all columns in a sparse matrix (in R).
Add to all columns motive: "Sample1_"
Result: Sample1_AAACGAACACGTTCGG
> class(matrix1)
[1] "dgCMatrix&...
0
votes
1answer
25 views
Row standardize a Sparse Matrix
I have the following sparse matrix;-
library(Matrix)
a <- sparseMatrix(i = c(1, 1, 2, 2, 3, 3, 3, 3, 4, 5, 5),
j = c(1, 2, 2, 3, 2, 3, 4, 5, 3, 1, 5), x = 1)
I want to divide ...
1
vote
2answers
32 views
Combining sparse and einsum to perform large sparse sum
I have a matrix A with shape=(N, N) and a matrix B with the same shape=(N, N).
I am constructing a matrix M using the following einsum (using the opt_einsum library):
M = oe.contract('nm,in,jm,pn,qm-&...
1
vote
0answers
28 views
how to store/retrieve/work with a sparse 3D matrix in python
this question was already asked here. However, that was 10 years ago; I'd like to bring it back, to see if there are new ways to store/retrieve/work with a 3D sparse matrix.
I know scipy.sparse still ...
1
vote
1answer
32 views
Scipy large sparse array dimensions MemoryError
I'm working with relatively large high-dimensional sparse arrays using scipy.sparse.
The actual data and row/column indices are no issue to store.
The problem is I end up up with things like
sp....
1
vote
1answer
29 views
How can I prevent OpenMDAO components from allocating large dense arrays for derivatives?
I have some OpenMDAO components that do math on 1D lists of x-coordinates and 1D lists of y-coordinates, outputting lists of distance and angle from each pair. Each computation depends on only a ...
0
votes
0answers
17 views
TensorFlow GradienTape crashes with InvalidArgumentError when working on sparse Tensors
I am encountering strange behavior when trying to evaluate the derivatives of a result obtained by sparse tensor operations. If I blow up all sparse inputs to dense before operating on them, the ...
1
vote
1answer
34 views
Julia - Using JuMP Triangular Matrix Variables in JuMP causes bad type performance
On the next MWE, @code_warntype returns bad performance type, ##1469::JuMP.Containers.SparseAxisArray.
using JuMP, Gurobi
function MWE(n)
m = Model(Gurobi.Optimizer)
@variable(m, x[i=1:n, ...
1
vote
0answers
27 views
Matrix transformation for eigenvalue problem and HPC
I have matrix Q
Q.shape is (2,2,n,n). Where n is k**2 , k > 1
So Q is block matrix:
Q[0,0].shape is (n,n)
Q[0,1].shape is (n,n)
Q[1,0].shape is (n,n)
Q[1,1].shape is (n,n)
Nested inside Q matrices ...
1
vote
1answer
37 views
How to iterate through all non zero values of a sparse matrix and normal matrix
I am using Julia and I want to iterate over the values of a matrix. This matrix can either be a normal matrix or a sparse matrix but I do not have the prior knowledge of that. I would like to create a ...
0
votes
0answers
21 views
Banded matrix version of Gaussian Elimination with no pivoting
I am new to Matlab and I kind of face challenges whenever I am writing codes. I have a Matlab code on Gaussian elimination without pivoting and I would like to implement a banded matrix version of it, ...
0
votes
0answers
7 views
Can sparse dictionary learning be used on multiple images?
I am kind of confused about sparse dictionary learning.
Now I am aware that sparse dictionary learning can be used to represent a particular image with a learned sparse dictionary. But how about there ...
2
votes
1answer
31 views
Using Pytorch how to define a tensor with indices and corresponding values
Problem
I have a list of indices and a list of values like so:
i = torch.tensor([[2, 2, 1], [2, 0, 2]])
v = torch.tensor([1, 2, 3])
I want to define a (3x3 for the example) matrix which contains the ...
1
vote
0answers
58 views
C++ Sparse Matrix Class
I have faced this C++ problem:
In numerical analysis and scientific computing, a sparse matrix or sparse array is a matrix in which most of the elements are zero. There is no strict definition how ...
0
votes
0answers
20 views
Using sparse coding on a 1D signal
I am trying to learn a sparse representation of a time series audio signal using make_sparse_coded_signal() from sklearn.datasets. I have been experimenting with make_sparse_coded_signal() but it ...
0
votes
0answers
29 views
how to provide support for sparse matrices?
I came up with the parallel implementation of the
stochastic matrix calculation ( e.g. for a transition probabilities matrix in a Markov chain )
see for
description
timing
.
Now I would like to ...
1
vote
1answer
50 views
Cholmod error 'out of memory' : Merging Seurat Objects
I am trying to merge Seurat class objects that contain transcriptome count data (sparse matrix). I am relatively new to R, so any help/solutions is appreciated. I have added a screenshot of the data ...
1
vote
1answer
41 views
Compute sum of power of large sparse matrix
Given a query vector (one-hot-vector) q with size of 50000x1 and a large sparse matrix A with size of 50000 x 50000 and nnz of A is 0.3 billion, I want to compute r=(A + A^2 + ... + A^S)q (usually 4 &...
2
votes
1answer
40 views
Scala Memory leak in fo/ yield
Hitting a memory issue with the below sparse matrix multiply routine.
Works fine for very large sparse very sparse matrices, but as the density of the matrices increases it grinds to a halt.
Looks ...
0
votes
0answers
16 views
Multiplication of sparse matrices in python
I have some code that I have written in R, which works for matrices. I have been asked to translate this into python to implement using sparse matrices.
I have two csr_matrices mat1 and mat2. At the ...
0
votes
0answers
19 views
Hpfilter statsmodels calculation
I can't understand how statsmodels(python lib) calculates hp-filter.
https://www.statsmodels.org/devel/generated/statsmodels.tsa.filters.hp_filter.hpfilter.html
■ Hpfilter (Hodrick–Prescott filter) ...
0
votes
1answer
55 views
Given the output of numpy.where(), how to acquire a list of list of the indices
I am searching a rather large matrix for zero elements and then want to then loop over the rows and inside each row, overall the indices where I found a zero.
numpy.where(M==0) gives me (np.array, np....
1
vote
1answer
37 views
simplity construction of sparse (transition) matrix
I am constructing a transition matrix from a n1 x n2 x ... x nN x nN array. For concreteness let N = 3, e.g.,
import numpy as np
# example with N = 3
n1, n2, n3 = 3, 2, 5
dim = (n1, n2, n3)
arr = np....
0
votes
1answer
15 views
Using a CSR Matrix in a multivariate random forest classification model
I am trying to utilize a CSR matrix as a variable to enhance my model. This matrix is derived from analyzing tf-idf metrics from string values in a pandas dataframe.
The series that the CSR matrix is ...
0
votes
0answers
43 views
Calculation mistakes of SparseLU in Eigen (C++)
Recently I found that in some certain circumstances, the SparseLU solver will get the wrong answer.
I give an example in the attachment, where "A.csv" is the coefficient matrix of the ...
0
votes
0answers
24 views
Dot product between transpose of sparse matrix and vector in python scipy
I know the function spmat.dot(vec) that returns the standard matrix vector 'dot' product between a sparse matrix spmat and a numpy array vec in scipy.
I need to compute transpose(spmat).dot(vec). Is ...
-1
votes
0answers
19 views
how to perform retain function in scipy.sparse matrices tf.sparse.retain
I am converting tensorflow code to numpy/scipy code.
I need an equivalent of tf.sparse.retain in the class scipy.sparse.
Thanks
2
votes
1answer
23 views
Computing a single element of the adjugate or inverse of a symbolic binary matrix
I'm trying to get a single element of an adjugate A_adj of a matrix A, both of which need to be symbolic expressions, where the symbols x_i are binary and the matrix A is symmetric and sparse. Python'...
1
vote
0answers
31 views
Converting MATLAB Binary file to Eigen Sparse Matrix
I am new to C++ and want to use the Eigen library to solve some sparse linear algebra problems. I have my arrays however they are MATLAB mat. files, and so I have loaded them into binary files like so:...
1
vote
0answers
32 views
Eigen: Modify Rows of Row-Major Sparse Matrix
I am using the Eigen library in C++ for solving sparse linear equations: Ax=b where, A is a square sparse matrix and b is a dense vector with ILU-preconditioned BiCGSTAB. I am initializing the matrix ...
0
votes
1answer
19 views
Trying to make a graph from a sparse matrix: not enough values to unpack (expected 2, got 0)
So I'm trying to make a graph with squares that are colored according to probability densities stored in the 7x7 matrix 'nprob'.
nprob = prob/sum
print(nprob.todense())
x,y = np.meshgrid(np.arange(0,...
0
votes
1answer
27 views
Why does my matrix with variable indices include only the first row?
So, I'm trying to make a graph with different colored squares where each one is colored according to the probability stored in my csc_matrix nprob, which is 7x7. The x and y in my graph are just ...
0
votes
0answers
61 views
How to create a sparse_matrix from a dataframe? Memory Error
I have a dataframe of size 78,000,000 rows x 14 columns. I want to get a sparse matrix from it for data training. To do this, I do the conversion using pd dummies. And I get a dataframe of 78,000,000 ...
1
vote
1answer
27 views
Load a sparse data array in python
I have multiple files composed of sampled data, each file contains x,y coordinates and the value of the variable. How can I possibly load this data as a sparse array (or matrix) where each sample is ...
0
votes
1answer
41 views
How to effieicently set specific rows and columns of a sparse matrix to zero in Eigen
I have the following section of the code that seems to take awhile to perform. Basically based on the value of an array index, if the value is 1 or higher, then I want to set that element in the ...
0
votes
0answers
28 views
How to perform indexing and slicing for sparse tensor
I'm trying to access elements of the sparse coo tensor.
i = [[0,2]]
v = [np.array([1,2,3,4]),[5,6,7,8]]
a = torch.sparse_coo_tensor(i,v,(4,4))
print(a.to_dense())
tensor([[1, 2, 3, 4],
[0, 0, ...