Questions tagged [covariance-matrix]

The tag has no usage guidance.

covariance-matrix
Filter by
Sorted by
Tagged with
1 vote
0 answers
19 views

Heteroscedasticity-consistent (robust) standard errors complemented by i) confidence intervals for beta, ii) Tolerance and iii) VIF values in R? [migrated]

In order to solve heteroscedasticity in my data, I ran a regression with heteroscedasticity-consistent ("robust") standard errors. I used SPSS, but to my current knowledge that is only able ...
0 votes
1 answer
34 views

cov2corr() for scipy sparse matrices

How do I make (big) sparse covariance matrices into sparse correlation matrices? Following the code for statsmodels.stats.moment_helpers.cov2corr(), if the covariance matrix isn't too big I can (...
1 vote
0 answers
26 views

Matérn covariance in R is returning matrices that are not positive definite [closed]

In R, I am trying to calculate a Matérn covariance matrix whose inputs are randomly created distance matrices. However, I end up getting covariance matrices that are not positive definite, which makes ...
0 votes
1 answer
44 views

Best fit of an increasing function that becomes constant

I am trying to fit a function to the following data. x_data = np.array([0.01 , 0.01871795, 0.0274359 , 0.03615385, 0.04487179, 0.05358974, 0.06230769, 0.07102564, 0.07974359, 0.08846154, ...
0 votes
0 answers
34 views

Covariance estimation using the Factor Model

The Factor Model, covariance estimation I'm currently working on replicating the factor covariance matrix estimation process in Python. However, I've encountered some doubts about my implementation. ...
0 votes
0 answers
22 views

Calculating variance-covariance matrix in R for Newick phylogeny

I am trying to calculate the variance-covariance matrix for a known tree but the ape function, vcv.phylo is not giving me expected results. library(ape) phy1 <- read.tree(text = "((a:6,b:6):...
2 votes
0 answers
33 views

Scikit-learn Gaussian Process Regressor returns 2d vector instead of 2d Covariance matrix

I'm fitting data (2d input, 2d output) to a Gaussian Process from Sklearn but when trying to get the covariance matrix I'm getting a 2d vector, not a matrix. For some examples, it works fine (returns ...
0 votes
0 answers
21 views

Difference between covariance using outer product and covariance using inner product?

I was going through a course, where in i am calculating covariance using inner product and outer product. For cov using inner product, I've used below code: inner_product = np.dot(...
0 votes
1 answer
46 views

How to check if a matrix is a var-covariance matrix as a constraint for optimization

I am doing an optimization where the parameters need to be estimated are variance-covariance matrix. And I am using differential_evolution from scipy for the optimation. def likelihood(params): p =...
0 votes
0 answers
37 views

Checking calculation of covariance matrix in R

I came across this formula in a text that says $S$ is the sample covariance matrix where $$S = \sum_{j=1}^n(\mathbf{X}_j - \bar{\mathbf{X}})(\mathbf{X}_j-\bar{\mathbf{X}})'$$, or from the source: ...
1 vote
1 answer
83 views

Cholesky factor. Matrix is not positive definite

I have a variance-covariance matrix as part of my optimization problem, and I want to get the Cholesky factor. The parameters that I am trying to estimate in the optimization are standard deviations ...
0 votes
1 answer
36 views

reconstruct Covariance matrix from dataset generated given that Covariance matrix (using Cholesky factorization)

Recalling that C = [email protected] using Cholesky factorization, I'm trying to repeat transformations described in this article with Python, but am having some misunderstanding & inability to return ...
0 votes
0 answers
43 views

Problem with a random effect with 2 levels in fitExtractVarPartModel() from the the variancePartition package?

I'm running a variance decomposition to understand how different predictors contribute to explaining the variance in a variable of interest. I'm using fitExtractVarPartModel from the variancePartition ...
3 votes
1 answer
110 views

Why is the covariance matrix from R's glm different to that from minitab's probit analysis

Using the dataset in the R script below. The output from R binomial probit glm vcov function is different to that from Minitab's probit analysis variance covariance table. R Documentation for vcov ...
1 vote
1 answer
106 views

Covariance deviation when using method L-BFGS-B of scipy.optimize.minimize

When assessing covariance matrix with scipy.optimize.minimize for a simple LLS problem, I have got significant deviation when using the method L-BFGS-B. After a lot of research, I have created a ...
0 votes
0 answers
22 views

Calculate square root of covariance matrix in Scala

I have a training dataset with some feature columns. I'm converting this dataset into a StandardScaler dataset and using that to generate a covariance matrix, and then calculating the square root of ...
0 votes
0 answers
11 views

Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) : NA/NaN/Inf in 'y'

I'm running the following command: cov.28.pc.raw <- PCs_Wcov$28 ##PCS_Wcov was obtained by: PCs_Wcov = llply(Wcov_all, eigen), where PCs_Wcov are the weighted averages of covariance matrices along ...
1 vote
3 answers
1k views

DataFrame to breeze DenseMatrix in spark using scala

I'm trying to convert a dataframe to a breeze dense matrix using scala. I couldn't find any built-in functions to do this, so here's what I'm doing. import scala.util.Random import breeze.linalg....
0 votes
0 answers
22 views

Calculate HC3 covariance matrix in Matlab

I am trying to calculate a HC3 covariance matrix to adjust for unequal variances as add in and update parameters of a model. Here is how I am calculating it in matlab: hat = X(1:t,:)...
0 votes
0 answers
64 views

Graphical lasso implementation with independent regularization per variable

I'm currently looking for a graphical lasso implementation allowing to specify different amount of regularization per variable. As precised in the original publication (End of page 6, Remark ; also ...
0 votes
1 answer
22 views

Horizontal legend for a covariance matrix plot

I am trying to construct a 2D perspective plot of a covariance matrix in R. A reprex code is below nrows <- 10 ncols <- 10 p <- nrows * ncols Qvariance <- 1 Qrho <- 0.8 alpha <- ...
2 votes
3 answers
40 views

r long to wide and covariance matrix

This is my dataset, df1 <- "ID t res 1 1 -1.5 1 2 -1.5 1 3 0.5 1 4 0.5 2 1 -0.5 2 2 ...
0 votes
0 answers
29 views

Why the result of Qt estimate of fitSSM function in KFAS package is negative?

I have this following model, I want to estimate the matrix of Ht and Qt using 200 random initialization using multivariate uniform distribution, then choosing the maximum likelihood. I try this ...
0 votes
1 answer
81 views

Lagged Covariance functions in R are extremely slow

I am trying to estimate the lagged covariance between two vector. I have used gsignal::xcov, forecast::Ccf and stats::ccf. My code has several nested loops so computing time is pilling up, with ...
0 votes
0 answers
70 views

Error message in R after conducting ANOVA

I want to conduct a mixed ANOVA in R with three factors (factor group and age_group are between subjects, factor test_interval is within subject) and used the aov-function of the "car" ...
0 votes
1 answer
189 views

Find Covariance and Correlation of Joint Probability Distribution in Python

If I'm given a joint distribution of 2 random variables say A and B, how would I find the covariance of A,B? (I have known to calculate E_X, E_Y, E_XY and apply formula, what I want to find library ...
0 votes
0 answers
50 views

Problem with Matlab multivariate normal distribution (mvncdf)

I want to create a normal distribution with a MU = 0 and SIGMA = cov(A); A is a 4000*5460 matrix. I get this error: SIGMA must be a square, symmetric, positive definite matrix. Since the error is ...
0 votes
1 answer
111 views

Wrong VCOV matrix from sandwich::vcovHC with observation counts in R

I am trying to estimate a weighted LM with repeated observations. Naturally, one can exploit this redundancy information to save space and memory. I am working on an application with 400k observations,...
0 votes
0 answers
60 views

Understanding variance calculations in scipy.optimise.curve_fit

I'm trying to use Scipy's scipy.optimise.curve_fit to calculate the parameters for a non-linear least squares fit of my data to a function. I have 2-3 response values for each value of my explanatory ...
0 votes
1 answer
75 views

why multiply covariance matrix by 2 in lmfit error calculation

When I looked into how the error calculation is done for lmfit when we fit functions, I found that the covariance matrix is calculated as the inverse of Hessian Matrix * 2. Then, the error on each ...
1 vote
1 answer
73 views

block diagonal covariance matrix by group of variable

If this is my dataset, 3 subjects, measured at two time points t1,t2 , at each time point each subject is measured twice ID Time Observation Y 1 t1 1 ...
1 vote
1 answer
164 views

ValueError: The truth value of a Index is ambiguous. Use a.empty, a.bool(), a.item(), a.any() [duplicate]

I am writing this code for streamlit deployment: if show_covariance_matrix: if not cov_matrix.columns.empty: fig = ff.create_annotated_heatmap(z=cov_matrix.values, x=cov_matrix.columns....
1 vote
1 answer
215 views

scipy.optimize.leastsq in Python not returning covariance matrix when fitting data

I am using optimize.leastsq to fit the data I have collected from a Mossbauer Spectroscopy experiment. The data was successfully fit, and the best fit parameters returned are good. But the second ...
0 votes
1 answer
151 views

Is it possible to have a covariance matrix without inverse using numpy?

I have the following problem: I need the inverse of the covariance matrix using the following data: x = [255, 239, 213] y = [255, 240, 245] I get the following covariance matrix using numpy.cov() ...
0 votes
2 answers
403 views

Plot existing covariance dataframe

I have computed a covariance of 26 inputs from another software. I have an existing table of the results. See image below: What I want to do is enter the table as a pandas dataframe and plot the ...
0 votes
0 answers
69 views

How to get r-squared using the covariance matrix output of scipy.optimize.curve_fit?

The scipy.optimize.curve_fit outputs covariance matrix, see scipy.optimize.curve_fit. I wish to get a single float description of the fitting result and I was use to compare r-squared, the coefficient ...
1 vote
0 answers
48 views

Error list' object cannot be coerced to type 'double'

I'm trying to get the "EvolvabilityMeans" (evolvability package) of covariance matrices from 14 species. First, I'm getting the list of covariance matrices and list of means (17 cranial ...
0 votes
0 answers
185 views

R: Computing covariance matrix of a large dataset

I'm trying to compute the covariance matrix of a very large image data matrix. I have tried both cov(data) and data %*% t(data)/ (nrow(t(data))-1) and ended up with a matrix of NaN values which ...
1 vote
1 answer
397 views

Stan Covariance Matrix

I am experimenting with stan and Gaussian Processes. After some errors I found out the root of everything is a strange behavior of the function cov_exp_quad. In particular I do not understand why it ...
0 votes
1 answer
171 views

How to get the within-group variance-covariance matrix in statsmodels MANOVA, Python?

I am trying to extract the within-group variance-covariance matrix to estimate the flatness in a MANOVA model in statsmodels using the following formula: I've been reading through the statsmodels ...
0 votes
0 answers
177 views

Matlab - determinant of covariance matrix coming out as zero

I'm trying to calculate the determinant of a 171x171 covariance matrix in matlab but the determinant is coming out as zero. I know the matrix is not singular because I am able to calculate the inverse ...
1 vote
0 answers
747 views

Using Eigen's SelfAdjointEigenSolver to get get the EigenVectors of a covariance matrix doesn't output expected values

I'm having trouble getting the proper eigenvectors from a covariance matrix. Every other tool I've tried (Hipparchus, numpy, and Wolfram Alpha) all have the same results. Note that the bolded values ...
0 votes
1 answer
504 views

Estimating parameter covariance matrix when fitting a Generalized Extreme Value (GEV) model using 'extRemes' in R

My question is similar to Error with fitting a Generalized Extreme Value (GEV) using `extRemes` in R?. However, I am fitting non-stationary Generalized Extreme Value (GEV) distribution, i.e., when the ...
0 votes
0 answers
90 views

Deriving data from a multivariate normal distribution according to a specific correlation matrix in r

I derive data from a multivariate normal distribution. I created the correlation structure I wanted by the sigma matrix. Since I take the standard deviation as 1, I expect the Sigma matrix and ...
0 votes
0 answers
63 views

SSM model Standard errors Matlab covariance matrix

Does anybody know please how to obtain standard errors of estimates parameter in SSM model matlab? or how to get them from covariance matrix?
0 votes
1 answer
211 views

Why isn't my Matlab code for a randomly generated covariance matrix making a positive definite matrix?

Here is my code. I'm getting an error that when I use chol(V) that V is not positive definite. I would think that by construction it must be positive definite. Any idea what's going wrong? % I want ...
7 votes
1 answer
2k views

How to use a lasso with the Vars package?

I'm trying to analyze a high dimensional data set (31 variables, 1100 observations) through a penalized vector autoregression. Since I'm using the techniques introduced by Diebold et. al (2019) to ...
1 vote
2 answers
407 views

Using the correlation matrix after a fit in Gnuplot

Say I need to fit some data to a parabola, and then perform some calculations involving the correlation matrix elements of the fit parameters: is there a way to use these parameters directly in ...
2 votes
0 answers
1k views

What is model.cov_params() in statsmodels?

I am unable to understand what the [cov_params][1] from a fitted statsmodel represents. I thought it would be the covariance matrix of the data but that does not seem to be the case. It is not even ...
3 votes
2 answers
428 views

Can't calculate np.cov() correctly

This question might be silly, but i couldn't find an explanation to that. I am coding the multivariate probability density function from scratch (for study purposes), and one of the things that i need ...