5

I have two sets of vectors x_i \in R^n and z_i \in R^m

I want to find a transformation matrix W such that W x_i approximates z_i,

i.e. I want to find W that minimizes: sum_i || W x_i − z_i ||^2

Is there a Python function that does this?

12
  • 2
    How many of each do you have? I think you're looking for least squares - is that right?
    – Joel
    Jan 16, 2015 at 9:03
  • I have 5000 vectors of each. x_i have dimension 300, and z_i dimension 800. yes, it is least squares.
    – Mostafa
    Jan 16, 2015 at 9:10
  • 1
    You're trying to use some mathematical formatting that Stack Overflow doesn't support at all... please make you question more readable. Jan 16, 2015 at 9:13
  • It's not clear from your question - are you familiar with numpy?
    – Joel
    Jan 16, 2015 at 9:22
  • no, it was a mistake to put numpy in labels
    – Mostafa
    Jan 16, 2015 at 9:28

1 Answer 1

4

Using this kronecker product identity it becomes a classic linear regression problem. But even without that, it is just the transpose of a linear regression setup.

import numpy as np
m, n = 3, 4
N = 100  # num samples

rng = np.random.RandomState(42)

W = rng.randn(m, n)
X = rng.randn(n, N)
Z_clean = W.dot(X)

Z = Z_clean + rng.randn(*Z_clean.shape) * .001

Using Z and X we can estimate W by solving argmin_W ||X^T W^T - Z^T||^2

W_est = np.linalg.pinv(X.T).dot(Z.T).T

from numpy.testing import assert_array_almost_equal
assert_array_almost_equal(W, W_est, decimal=3)
2
  • Hi, Do you mind elaborating a little on this step : Z = Z_clean + rng.randn(*Z_clean.shape) * .001 Feb 24, 2016 at 14:12
  • I generate the clean target Z_clean according to the model. Then I add noise (the line you are inquiring about does this) in order to show that the method works even if the data are slightly perturbed. The noise is Gaussian because a least-squares estimate implicitly assumes Gaussian noise.
    – eickenberg
    Feb 24, 2016 at 20:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.