0

I have a dataset that is being used to compute(approximate) the parameters of a non-linear function.

The raw data points are spread out in time and currently my solver is able to compute the best set of parameters that model the function for data items in a given period of time. The accuracy of the function approximation improves as I incorporate a larger data set. At the same time however, I don't want data items that are too old to largely effect the function approximation. I am now planning to use data items that fall within a pre-defined window in time. This predefined window will move as time progresses, incorporating new data items and discarding old ones. However to include or exclude data elements I always have to start the process from the beginning with the modified data set, a process which is time consuming and not suited for real time-time operation.

The problem I am trying to tackle is how to incorporate learning from additional data items into the approximated function without having to go through the entire original data set. An initial idea is to weight the function parameters learned from each subset of data by the ratio of the total data items in the subset to the total data items in all the subsets. Can anybody think of a better approach? A hint toward any possible solution would be greatly appreciated.

1 Answer 1

0

Borrowing from some time series techniques, one simple (heuristic) approach is to use exponential smoothing of your model parameters where you calculate weighted average of the newly learned parameters (based on recent data) and the older parameters (value of the weight would have to be tuned using some sort of cross-validation / backtesting). This usually works quite well if the signal-to-noise ratio does not change dramatically in the new data as a function of time.

Another approach is to impose a "prior" for your model parameters based on earlier data - one of the simplest ways to do that (which does not require full bayesian learning) is to add a quadratic penalty to your loss function that penalizes for deviating from the older parameter values (with the penalty coefficient tuned using cross-validation/backtesting) Some care should be taken to make sure that the variance-covariance matrix of the older model parameters is taken into account when constructing the penalty. This is roughly equivalent to imposing Gaussian prior based on older model parameters.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.