Does anyone know if there is any existing package in python to train loglinear model? I have a dataset with 2000 variables and 1000 records. I am looking to use loglinear model to estimate frequencies.
2 Answers
If you use an old version of SciPy (namely 0.10 or before), you can use scipy.maxentropy
(in NLP, MaxEnt = Maximum Entropy Modeling = Log-Linear models). The module got removed it from SciPy when version 0.11.0 got released, the SciPy team then advised to use sklearn.linear_model.LogisticRegression as a replacement (note that both log-linear models and logistic regressions are examples of generalized linear models, in which the relationship between a linear predictor).
Example using SciPy's maxentropy module (removed in SciPy 0.11.0):
#!/usr/bin/env python
""" Example use of the maximum entropy module:
Machine translation example -- English to French -- from the paper 'A
maximum entropy approach to natural language processing' by Berger et
al., 1996.
Consider the translation of the English word 'in' into French. We
notice in a corpus of parallel texts the following facts:
(1) p(dans) + p(en) + p(a) + p(au cours de) + p(pendant) = 1
(2) p(dans) + p(en) = 3/10
(3) p(dans) + p(a) = 1/2
This code finds the probability distribution with maximal entropy
subject to these constraints.
"""
__author__ = 'Ed Schofield'
__version__= '2.1'
from scipy import maxentropy
a_grave = u'\u00e0'
samplespace = ['dans', 'en', a_grave, 'au cours de', 'pendant']
def f0(x):
return x in samplespace
def f1(x):
return x=='dans' or x=='en'
def f2(x):
return x=='dans' or x==a_grave
f = [f0, f1, f2]
model = maxentropy.model(f, samplespace)
# Now set the desired feature expectations
K = [1.0, 0.3, 0.5]
model.verbose = True
# Fit the model
model.fit(K)
# Output the distribution
print "\nFitted model parameters are:\n" + str(model.params)
print "\nFitted distribution is:"
p = model.probdist()
for j in range(len(model.samplespace)):
x = model.samplespace[j]
print ("\tx = %-15s" %(x + ":",) + " p(x) = "+str(p[j])).encode('utf-8')
# Now show how well the constraints are satisfied:
print
print "Desired constraints:"
print "\tp['dans'] + p['en'] = 0.3"
print ("\tp['dans'] + p['" + a_grave + "'] = 0.5").encode('utf-8')
print
print "Actual expectations under the fitted model:"
print "\tp['dans'] + p['en'] =", p[0] + p[1]
print ("\tp['dans'] + p['" + a_grave + "'] = " + str(p[0]+p[2])).encode('utf-8')
# (Or substitute "x.encode('latin-1')" if you have a primitive terminal.)
Other ideas: http://homepages.inf.ed.ac.uk/lzhang10/maxent.html
I'm not sure if this solves your problem as you mentioned "machine-learning" and it's not clear what kind of data you have. But since you also mentioned "prediction" and "estimate frequencies" I'll guess that interpolation could be helpful. In this case you can have a look at scipy.interpolate
.
The Rbf
interpolator is "A class for radial basis function approximation/interpolation of n-dimensional scattered data...". It supports the following functions:
'multiquadric': sqrt((r/self.epsilon)**2 + 1)
'inverse': 1.0/sqrt((r/self.epsilon)**2 + 1)
'gaussian': exp(-(r/self.epsilon)**2)
'linear': r
'cubic': r**3
'quintic': r**5
'thin_plate': r**2 * log(r)
-
Thanks for the reply. I guess I am trying to figure out if there is any easier way to implement the model in this paper users.cis.fiu.edu/~lzhen001/activities/KDD_USB_key_2010/docs/… May 1, 2013 at 5:39