4

I want to implement a kernel ridge regression that also works within MLJ. Moreover, I want to have the option to use either feature vectors or a predefined kernel matrix as in Python sklearn.

When I run this code

const MMI = MLJModelInterface    
MMI.@mlj_model mutable struct KRRModel <: MLJModelInterface.Deterministic
        mu::Float64 = 1::(_ > 0)
        kernel::String = "linear"
end
function MMI.fit(m::KRRModel,verbosity::Int,K,y)
    K = MLJBase.matrix(K)
    fitresult = inv(K+m.mu*I)*y
    cache = nothing
    report = nothing
    return (fitresult,cache,report)
end

   
N = 10
K = randn(N,N)
K = K*K
a = randn(N)
y = K*a + 0.2*randn(N)
m = KRRModel()
kregressor = machine(m,K,y)
cv = CV(; nfolds=6,  shuffle=nothing, rng=nothing)
evaluate!(kregressor, resampling=cv, measure=rms, verbosity=1)

the evaluate! function evaluates the machine on different subsets of rows of K. Due to the Representer Theorem, a kernel ridge regression has a number of nonzero coefficients equal to the number of samples. Hence, a reduced size matrix K[train_rows,train_rows] can be used instead of K[train_rows,:].

To denote I'm using a kernel matrix I'd set m.kernel = "" . How do I make evaluate! select the columns as well as the rows to form a smaller matrix when m.kernel = ""?

This is my first time using MLJ and I'd like to make as few modifications as possible.

1
  • I guess I could easily do this by manually calling fit! with the smaller kernel matrix, but I want to get the whole MLJ pipeline working Commented Dec 17, 2020 at 18:15

1 Answer 1

0

Quoting the answer I got on the Julia Discourse from @ablaom

The intended use of evaluate! is to estimate the generalisation error associated with some supervised learning model, by subsampling observations, as in cross-validation, a common use-case. I’m afraid there is no natural way for evaluate! do feature subsampling.

https://alan-turing-institute.github.io/MLJ.jl/dev/evaluating_model_performance/

FYI: There is a version of kernel regression implementing the MLJ model interface, namely kernel partial least squares regression from the package https://github.com/lalvim/PartialLeastSquaresRegressor.jl

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.