I am using the LMNN
module from scikit-learn metric_learning
(http://contrib.scikit-learn.org/metric-learn/index.html), and I am attempting to recover the linear transformation matrix (L.T
) from the learned Mahalanobis (M
) matrix.
The reason I am trying to recover this linear transformation is that I am fitting my dataset using cloud compute, but am testing it on a local machine. This means I can not save or recover the LMNN
model after fitting on cloud compute, but I can save the learned M matrix and use a decomposition to find the learned linear transformation. I can then apply that learned linear transformation to my test sets on a local machine.
The problem is that I can't seem to reconcile the results from the LMNN
module's built in transformation with the learned linear transformation from the decomposed M
matrix. Here's an example:
import numpy as np
from metric_learn import LMNN
from sklearn.datasets import load_iris
iris_data = load_iris()
X = iris_data['data']
Y = iris_data['target']
lmnn = LMNN(k=5, learn_rate=1e-6)
X_transformed = lmnn.fit_transform(X, Y)
M_matrix = lmnn.get_mahalanobis_matrix()
array([[ 2.47937397, 0.36313715, -0.41243858, -0.78715282],
[ 0.36313715, 1.69818843, -0.90042673, -0.0740197 ],
[-0.41243858, -0.90042673, 2.37024271, 2.18292864],
[-0.78715282, -0.0740197 , 2.18292864, 2.9531315 ]])
# cholesky decomp of M_matrix
eigvalues, eigcolvectors = np.linalg.eig(M_matrix)
eigvalues_diag = np.diag(eigvalues)
eigvalues_diag_sqrt = np.sqrt(eigvalues_diag)
L = eigcolvectors.dot(eigvalues_diag_sqrt.dot(np.linalg.inv(eigcolvectors)))
L_transpose = np.transpose(L)
L_transpose.dot(L) # check to confirm that matches M_matrix
array([[ 2.47937397, 0.36313715, -0.41243858, -0.78715282],
[ 0.36313715, 1.69818843, -0.90042673, -0.0740197 ],
[-0.41243858, -0.90042673, 2.37024271, 2.18292864],
[-0.78715282, -0.0740197 , 2.18292864, 2.9531315 ]])
# test fit_transform() vs. transform() using LMNN functions
lmnn.transform(X[0:4, :])
array([[8.2487 , 4.41337015, 0.14988465, 0.52629361],
[7.87314906, 3.77220291, 0.36015873, 0.525688 ],
[7.59410008, 4.03369392, 0.17339877, 0.51350962],
[7.41676205, 3.82012155, 0.47312948, 0.68515535]])
X_transformed[0:4, :]
array([[8.2487 , 4.41337015, 0.14988465, 0.52629361],
[7.87314906, 3.77220291, 0.36015873, 0.525688 ],
[7.59410008, 4.03369392, 0.17339877, 0.51350962],
[7.41676205, 3.82012155, 0.47312948, 0.68515535]])
# test manual transform of X[0:4, :]
X[0:4, :].dot(L_transpose)
array([[8.22608756, 4.45271327, 0.24690081, 0.51206068],
[7.85071271, 3.81054846, 0.45442718, 0.51144826],
[7.57310259, 4.06981377, 0.26240745, 0.50067674],
[7.39356544, 3.85511015, 0.55776916, 0.67615584]])
As seen above, the first four rows of the original dataset X[0:4, :]
when transformed by the LMNN
module (using either fit_transform(X, Y)
or transform(X[0:4, :])
give different results from the manual transformation.
- I believe my decomposition of the M matrix is correct as I can replicate the M matrix using
L.T.dot(L)
. - The learned linear transformation is
L.T
as per the github code: https://github.com/scikit-learn-contrib/metric-learn/blob/master/metric_learn/base_metric.py
class MetricTransformer(six.with_metaclass(ABCMeta)):
@abstractmethod
def transform(self, X):
"""Applies the metric transformation.
Parameters
----------
X : (n x d) matrix
Data to transform.
Returns
-------
transformed : (n x d) matrix
Input data transformed to the metric space by :math:`XL^{\\top}`
class MahalanobisMixin(six.with_metaclass(ABCMeta, BaseMetricLearner,
MetricTransformer)):
r"""Mahalanobis metric learning algorithms.
Algorithm that learns a Mahalanobis (pseudo) distance :math:`d_M(x, x')`,
defined between two column vectors :math:`x` and :math:`x'` by: :math:`d_M(x,
x') = \sqrt{(x-x')^T M (x-x')}`, where :math:`M` is a learned symmetric
positive semi-definite (PSD) matrix. The metric between points can then be
expressed as the euclidean distance between points embedded in a new space
through a linear transformation. Indeed, the above matrix can be decomposed
into the product of two transpose matrices (through SVD or Cholesky
decomposition): :math:`d_M(x, x')^2 = (x-x')^T M (x-x') = (x-x')^T L^T L
(x-x') = (L x - L x')^T (L x- L x')`
- What am I missing here?
Thanks!
numpy.linalg.svd
instead ofnp.linalg.eig
?