3

Does anyone know if there is any existing package in python to train loglinear model? I have a dataset with 2000 variables and 1000 records. I am looking to use loglinear model to estimate frequencies.

2 Answers 2

3

If you use an old version of SciPy (namely 0.10 or before), you can use scipy.maxentropy (in NLP, MaxEnt = Maximum Entropy Modeling = Log-Linear models). The module got removed it from SciPy when version 0.11.0 got released, the SciPy team then advised to use sklearn.linear_model.LogisticRegression as a replacement (note that both log-linear models and logistic regressions are examples of generalized linear models, in which the relationship between a linear predictor).

Example using SciPy's maxentropy module (removed in SciPy 0.11.0):

#!/usr/bin/env python

""" Example use of the maximum entropy module:

    Machine translation example -- English to French -- from the paper 'A
    maximum entropy approach to natural language processing' by Berger et
    al., 1996.

    Consider the translation of the English word 'in' into French.  We
    notice in a corpus of parallel texts the following facts:

        (1)    p(dans) + p(en) + p(a) + p(au cours de) + p(pendant) = 1
        (2)    p(dans) + p(en) = 3/10
        (3)    p(dans) + p(a)  = 1/2

    This code finds the probability distribution with maximal entropy
    subject to these constraints.
"""

__author__ =  'Ed Schofield'
__version__=  '2.1'

from scipy import maxentropy

a_grave = u'\u00e0'

samplespace = ['dans', 'en', a_grave, 'au cours de', 'pendant']

def f0(x):
    return x in samplespace

def f1(x):
    return x=='dans' or x=='en'

def f2(x):
    return x=='dans' or x==a_grave

f = [f0, f1, f2]

model = maxentropy.model(f, samplespace)

# Now set the desired feature expectations
K = [1.0, 0.3, 0.5]

model.verbose = True

# Fit the model
model.fit(K)

# Output the distribution
print "\nFitted model parameters are:\n" + str(model.params)
print "\nFitted distribution is:"
p = model.probdist()
for j in range(len(model.samplespace)):
    x = model.samplespace[j]
    print ("\tx = %-15s" %(x + ":",) + " p(x) = "+str(p[j])).encode('utf-8')


# Now show how well the constraints are satisfied:
print
print "Desired constraints:"
print "\tp['dans'] + p['en'] = 0.3"
print ("\tp['dans'] + p['" + a_grave + "']  = 0.5").encode('utf-8')
print
print "Actual expectations under the fitted model:"
print "\tp['dans'] + p['en'] =", p[0] + p[1]
print ("\tp['dans'] + p['" + a_grave + "']  = " + str(p[0]+p[2])).encode('utf-8')
# (Or substitute "x.encode('latin-1')" if you have a primitive terminal.)

Other ideas: http://homepages.inf.ed.ac.uk/lzhang10/maxent.html

1

I'm not sure if this solves your problem as you mentioned "machine-learning" and it's not clear what kind of data you have. But since you also mentioned "prediction" and "estimate frequencies" I'll guess that interpolation could be helpful. In this case you can have a look at scipy.interpolate.

The Rbf interpolator is "A class for radial basis function approximation/interpolation of n-dimensional scattered data...". It supports the following functions:

'multiquadric': sqrt((r/self.epsilon)**2 + 1) 
'inverse':      1.0/sqrt((r/self.epsilon)**2 + 1)
'gaussian':     exp(-(r/self.epsilon)**2)
'linear':       r 
'cubic':        r**3 
'quintic':      r**5
'thin_plate':   r**2 * log(r)
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.