1

I'm wanting to determine whether a time series is mean-reverting or not, but I'm running into some issues when calculating the Hurst exponent. It's supposed to print 0.5-ish, but instead I get a "nan". All help would be appreciated.

I get the following error/warning:

RuntimeWarning: divide by zero encountered in log
  poly = polyfit(log(lags), log(tau), 1)

Below is the code I'm working on.

import statsmodels.tsa.stattools as ts
from datetime import datetime

from pandas_datareader import DataReader
security = DataReader("GOOG", "yahoo", datetime(2000,1,1), datetime(2013,1,1))
ts.adfuller(security['Adj Close'], 1)



from numpy import cumsum, log, polyfit, sqrt, std, subtract
from numpy.random import randn

def hurst(ts):
    """Returns the Hurst Exponent of the time series vector ts"""

    lags = range(2, 100)

    tau = [sqrt(std(subtract(ts[lag:], ts[:-lag]))) for lag in lags]

    poly = polyfit(log(lags), log(tau), 1)


    return poly[0]*2.0


gbm = log(cumsum(randn(100000))+1000)
mr = log(randn(100000)+1000)
tr = log(cumsum(randn(100000)+1)+1000)

print ("Hurst(GBM):   %s" % hurst(gbm))
print ("Hurst(MR):    %s" % hurst(mr))
print ("Hurst(TR):    %s" % hurst(tr))
print ("Hurst(SECURITY):  %s" % hurst(security['Adj Close']))



print ("Hurst(GBM):   %s" % hurst(gbm))
print ("Hurst(MR):    %s" % hurst(mr))
print ("Hurst(TR):    %s" % hurst(tr))
print ("Hurst(SECURITY):  %s" % hurst(security['Adj Close']))
Hurst(GBM):   0.5039604262314196
Hurst(MR):    -2.3832407841923795e-05
Hurst(TR):    0.962521148986032
Hurst(SECURITY):  nan
__main__:11: RuntimeWarning: divide by zero encountered in log
2
  • One or more of the values in ts are zero. The warning and result you see is NumPy attempting to take the natural logarithm of zero, and setting the result to Not a Number, nan.
    – 9769953
    May 8, 2020 at 20:56
  • Alternatively, since you're doing std(subtract(ts[lag:], ts[:-lag])), that result may be zero instead of ts (more likely even than a value in ts being zero), and thus one or more values in tau are zero, with the same warning and final nan result.
    – 9769953
    May 8, 2020 at 20:58

2 Answers 2

1

I had the same problem when sending Series as the ts argument. All you have to do is send a List not a Series or:

def hurst(ts):
    """Returns the Hurst Exponent of the time series vector ts"""
    ts = ts if not isinstance(ts, pd.Series) else ts.to_list()
    lags = range(2, 100)
    tau = [sqrt(std(subtract(ts[lag:], ts[:-lag]))) for lag in lags]
    poly = polyfit(log(lags), log(tau), 1)
    return poly[0]*2.0

NaN values might be an issue as well, I would check if is ok to dropna() before to_list()

1
  • Thank you for the input! I found a way around it. When I isolated the closing prices or made a excel file with a single column, I could run the data through the function with no problem. Aug 16, 2020 at 23:02
0

The root cause is that the Series[<slice>] syntax returns the corresponding index for each slice, and the - operator works on per-index equality (not actual location).

Example:

s = pd.Series(range(5))
s[2:] - s[:-2]
=>
0    NaN
1    NaN
2    0.0
3    NaN
4    NaN
dtype: float64

Clearly, that's not what we expected. To see why we can use concat to create a row-by-row dataframe of s[2:], s[:-2], respectively.

pd.concat([s[2:], s[:-2]], axis=1)
=>
    0   1
0   NaN 0.0
1   NaN 1.0
2   2.0 2.0
3   3.0 NaN
4   4.0 NaN

Given this input the result of the tau = equation in the hurst function is a list of (mostly) nan values.

The solution to work with Series natively is to use Series.shift() instead of array slicing:

def hurst(ts):
  ... 

  # Calculate the array of the variances of the lagged differences
  tau = [sqrt((ts - ts.shift(-lag)).std()) for lag in lags]

  ...

Alternatively, pass the Series.values to the original function, which passes a numpy array.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.