1

For MRE:

m = 100
X = 6*np.random.rand(m,1)-3
y = 0.5*X**2 + X+2 + np.random.randn(m,1)

lin_reg = LinearRegression()
lin_reg.fit(X,y)
y_pred_1 = lin_reg.predict(X)
y_pred_1 = [_[0] for _ in y_pred_1]

Plotting (X,y) on the graph works fine. Plotting (X, y_pred_1) gives me a line of best fit. Now since my y value above is created using X to the power of 2 thus it would look like a parabola.

So best fitting line would not be linear in this case but polynomaial with degree 2.

So I do:

poly_features = PolynomialFeatures(degree=2, include_bias=False)
X_poly_2 = poly_features.fit_transform(X)

poly_reg_2 = LinearRegression()
poly_reg_2.fit(X_poly_2, y)

y_pred_2 = poly_reg_2.predict(X_poly_2)
y_pred_2 = [_[0] for _ in y_pred_2]

and plot it on my graph which gives me something like a parabola but contains too much line. here is what I get when I plot points, predicting line of 1-degree, prediction line of 2-degree.

Using ploty:

import plotly.graph_objects as go
plot_X = [_[0] for _ in X.tolist()]
plot_y = [_[0] for _ in y.tolist()]

fig = go.Figure()
fig.add_trace(
    go.Scatter(
        x = plot_X,
        y = plot_y,
        mode="markers"
    )
)

fig.add_trace(
    go.Scatter(
        x = plot_X,
        y = y_pred_1,
        name="degree = 1"
    )
)

fig.add_trace(
    go.Scatter(
        x = plot_X,
        y = y_pred_2,
        name="degree = 2"
    )
)

fig.show()

which outputs enter image description here

What am I doing wrong?

Out of curiosity why does sklearn use linear regression to predict non-linear things like parabola in my case?

Also if I run poly_reg_2.coef_ it gives me array([[0.99366804, 0.45225746]]) how would I interpret this?

y = 0.99366804x + 0.45225746x was what I've thought but then it would not draw parabola how do you know which coefficient to raise to a power of 2 and which one to keep it degree =1?

EDIT: when I plot using

fig.add_trace(
    go.Scatter(
        x = plot_X,
        y = y_pred_2,
        name="degree = 2",
        mode="markers"
    )
)

adding mode parameter and setting it to marker which creates a scatterplot it seems to show work fine but in scatterplot.

2 Answers 2

2

I had the same problem. This is my way to solve it.

x_predict = np.linspace(-3, 3, 100)
y_predict = lin_reg.predict(poly_features.transform(x_predict .reshape(-1, 1)))
plt.plot(x_predict , y_predict)
plt.plot(X, y, 'bo')
plt.show()
1

It seems you have error in X_poly_2 before enter into the LR. It seems X_poly_2 should be changed to

X_poly_2=X_poly_2[:,0].reshape(100,1)
3
  • Thanks but could you explain what it is doing? I've thought it changes to 100x1 matrix but some values seems to go missing.
    – haneulkim
    Feb 13, 2020 at 3:41
  • Also it does not seem to work. When I to that before I fit my model it does not output anything on the graph
    – haneulkim
    Feb 13, 2020 at 3:44
  • Could you compare X and X_poly_2, then you can get the difference.
    – puhuk
    Feb 13, 2020 at 3:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.