Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.

As in sklearn, LogisticRegression(short for LR) has not direct method for solving weighted LR, so i pass to SGDClassifier(SGD).

As with my experiment: i generate data follow LR distribution with parametre intercept=0, beta=2. And run LR and SGD to estimate them. To compare this two, i set the same penalty parameter(my original idea is setting them to 0, but as they can't be setted to 0, i give big C for LR and small alpha for SGD )

As i see, LR almost do well for estimation, but it's difficult to setting the parametre for SGD. The main problem is the choice of eta0 and the learning_rate: 'constant' (too slow), 'optimal' or 'invscaling'.

My idea is watching the loss function, if it's likely to go down, increase the n_iter. if it go down too slow, increase the eta0. But

1.how to return the value of loss function for each epoch, i see them by change verbose to 1, but i don't know how to get return the value. (may be partiel_fit?)

2.is there more intelligent (automatique) way to this work? if not i should relance the training process many times And more complicated if i use cross validation

Thanks you for all your advice. If i was not clear, please let me know.

P.S. the code in Python require indent block, as i'm new to stackoverflow, i don't know how to do this, so if you want to execute de code, please add indented block after the def.

import random
import numpy as np
from sklearn.linear_model import LogisticRegression,SGDClassifier

def simule_logistic(n):
    beta=0.2
    x=[]
    seuil=[]
    for i in range(n):
        x.append(random.normalvariate(1, 2))
        seuil.append(random.uniform(0, 1))

    x=np.array(x)
    seuil=np.array(seuil)
    p=1.0/(1+ np.exp(-x*beta))


    y=[]
    for i in range(n):
        if p[i]<seuil[i]:
            y.append(0)
        else:
            y.append(1)
    y=np.array(y)

    return x, p,y


if __name__=='__main__':

    n=100000
    x,p,y=simule_logistic(n)
    x=x.reshape((n,1))
    print x.shape
    print y.shape
    l=LogisticRegression(C=1000000,penalty='l1')
    l.fit(x,y)
    sgd=SGDClassifier(n_iter=100,n_jobs=1, loss='log',alpha=1.0/1000000,l1_ratio=1,learning_rate='optimal',eta0=0.01)
    print sgd
    sgd.fit(x,y)


    #methode regression

    print 'l',l.coef_
    print l.intercept_
    print 'sgd',sgd.coef_
    print sgd.intercept_
share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.