I am trying for quite long time to do the following in Python, for a new app that I am currently working on:
Inputs:
list_of_points
- n sample points x. Each has d dimensions (each is represented as a list), where d is assumed to be large (the larger the better).list_of_values
- n values "hidden function" f. That is,list_of_values[i] = f(list_of_points[i]) + noise
domain
- the d-minsional range on which f is defined, and thuslist_of_points
are also in this domain. The domain can possibly be the whole R^d.
Output: list of points that are global maxima of f on domain
.
In order to filter the noise, I decided to use regression to estimate f, and only then to look for its maxima points.
Summarizing the above to a pseudo-code, we get:
def filter_noise_and_return_global_maxima(list_of_points, list_of_values, domain, degree_of_regression_polynomial):
f = polynomial_regression(list_of_points, list_of_values, degree = degree_of_regression_polynomial)
return f.global_maxima(domain)
Now I was trying to figure out how to do that, and I could not find any way to do it, especially if I want the code to run in a reasonable time (considering that the dimension d is rather big).
For the polynomial regression I found the following in the web:
from sklearn.preprocessing import PolynomialFeatures
from sklearn import linear_model
X = [[0.44, 0.68], [0.99, 0.23]]
vector = [109.85, 155.72]
predict= [0.49, 0.18]
poly = PolynomialFeatures(degree=2)
X_ = poly.fit_transform(X)
predict_ = poly.fit_transform(predict)
clf = linear_model.LinearRegression()
clf.fit(X_, vector)
print(clf.predict(predict_))
But unfortunately this raises errors. Also, I am not sure how to produce the polynomial for sympy that is used below.
Finally, for finding the global maxima I found this:
from sympy.calculus.util import *
x = symbols('x')
f = (x**3 / 3) - (2 * x**2) - 3 * x + 1
print(minimum(f, x, ivl))
print(maximum(f, x, ivl))
print(stationary_points(f, x, ivl))
But I am not sure if choosing a symbolic computatin is a good choise as I deal with a large dimension, and also I didn't realize how to use it for the multivariate case. For example, the folloing does not work f = x[0]**2+ 2*x[1]
.
list_of_points
?list_of_points=[[1,2,3.2],[2,4.2, 5.1], [-1,0.1, 0.9]] list_of_values=[4,5,6], domain=[[-10,10], [-10,10], [-10, 10]]#-10<=x,y,z<=10, degree_of_regression_polynomial=1
p1=[1,2,3.2], p2=[2,4.2, 5.1], p3=[-1,0.1, 0.9]
and their corresponding values are 4, 5 and 6 respectively. Was this correct? We will create a model that will fit these values. From the given p1 with dimension 3 plus its value we are now dealing with a 4D data is this right? What exactly isOutput: list of points that are global maxima of f on domain.
Do you mean that after building our model, we will search the point with a maximum value?