0

I need to compute rolling window regressions in Python where the standard errors are corrected for HAC (Newey-West, 1987). I know that statsmodels has a function for rolling window regressions (Rolling Regression), but the standard errors cannot be corrected for HAC in this function. Therefore I have defined my own function.

I have a panel data set with 108,768 rows (410 unique funds) and the structure looks as follows:

funds = pd.DataFrame({
    "Fund": ["A", "A", "A", "A", "B", "B", "B", "B"], 
    "Excess_Return": [np.NaN, 0.172, 0.0465, 0.039, 0.003995, -0.022139, 0.009518, 0.03233],
    "Regression_Constant": [1,1,1,1,1,1,1,1], 
    "RMRF": [0.0118,0.0557,0.0129,0.0403,0.0118,0.0557,0.0129,0.0403], 
    "SMB": [0.0445,0.1838,-0.1539,-0.0496,0.0445,0.1838,-0.1539,-0.0496], 
    "HML": [-0.0189,-0.0981,0.0823,0.0725,-0.0189,-0.0981,0.0823,0.0725], 
    "RMW": [-0.0629,-0.1876,0.1182,0.0767,-0.0629,-0.1876,0.1182,0.0767], 
    "CMA": [0.0474,-0.0035,-0.0161,0.0562,0.0474,-0.0035,-0.0161,0.0562]})

The function is defined as follows:


min_t = 30

t = 36

def process(x):
    if x["Excess_Return"].count() >= min_t:
        reg = smf.ols("Excess_Return ~ RMRF + SMB + HML + RMW + CMA", data = x).fit(cov_type="HAC", cov_kwds={"maxlags":1})
        return [
            reg.params[0],
            reg.params["RMRF"],
            reg.params["SMB"],
            reg.params["HML"],
            reg.params["RMW"],
            reg.params["CMA"],
            # tvalues
            reg.tvalues[0],
            reg.tvalues["RMRF"],
            reg.tvalues["SMB"],
            reg.tvalues["HML"],
            reg.tvalues["RMW"],
            reg.tvalues["CMA"],

        ]
    # Else return NaN
    return [np.nan] * 10

  

To run the function and get the results in a data frame i run the following command:

df_1 = funds.join(
    # join new DataFrame back to original
    pd.DataFrame(
        (process(x) for x in funds.rolling(t)),
        columns=["alpha", "Beta_RMRF", "Beta_SMB", "Beta_HML", "Beta_RMW", "Beta_CMA",
                 "t_alpha", "t_RMRF", "t_SMB", "t_HML", "t_RMW", "t_CMA"]
    )
)


This returns the correct t-stats of the regressions. However, since I have multiple funds, I need to group the funds by their name, i.e. that the returns of fund A are not taken into account when running regressions for fund B. Does anybody have a solution how to rearrange the code?

2
  • It does not work that way. (1) you miss the return output of the function. (2) if you want to do the rolling regression then I suggest you to visit this link: pandas.pydata.org/docs/reference/api/…
    – PTQuoc
    Jul 29, 2022 at 3:11
  • Many thanks for your reply. The question is now edited. The function is now working properly. The last problem I have is the grouping of funds. Do you have an idea regarding this issue?
    – MF1992
    Aug 17, 2022 at 14:10

1 Answer 1

0

Below is a working example with RollingOLS from statsmodels. The inspiration is from the answer to this question on Rolling OLS Regressions and Predictions by Group.

This can easily be modified for your panel data to perform a rolling window regression. Specify the covariance structure in model.fit(). See https://www.statsmodels.org/dev/generated/statsmodels.regression.rolling.RollingOLS.fit.html#statsmodels.regression.rolling.RollingOLS.fit.

from statsmodels.regression.rolling import RollingOLS
from statsmodels.tools.tools import add_constant
import statsmodels.api as sm
import pandas as pd
import numpy as np

data = sm.datasets.grunfeld.load()
df_grunfeld = pd.DataFrame(data.data)
df_grunfeld.set_index(['firm'], append=True, inplace=True)

# Simple Model
# $$invest = \beta_0 + \beta_1 value$$

def invest_params(df_gf, intercept=False):

    """
    Function to operate on the data of a single firm.
    Assumes df_gf has the columns 'invest' and 'value' available.
    Returns a dataframe containing model parameters
    """

    # we should have at least k + 1 observations
    min_obs = 3 if intercept else 2
    wndw = 8 

    # if there are less than min_obs rows in df_gf, RollingOLS will throw an error
    # Instead, handle this case separately

    if df_gf.shape[0] < min_obs:
        cols = ['coef_intercept', 'coef_value'] if intercept else ['coef_value']
        return pd.DataFrame(index=df_gf.index, columns=cols)

    y = df_gf['invest']
    x = add_constant(df_gf['value']) if intercept else df_gf['value']

    model = RollingOLS(y, x, expanding=True, min_nobs=min_obs, window=wndw).fit()

    parameters = model.params
    params_shifted = model.params.shift(1)
    mse = model.mse_resid
    parameters['invest_hat'] = (parameters.mul(add_constant(df_gf['value']), axis=0)\
                                .sum(axis=1, min_count=1)).to_frame('invest_hat')

    parameters['invest_hat_shift'] = (params_shifted.mul(add_constant(df_gf['value']), axis=0)\
                                      .sum(axis=1, min_count=1)).to_frame('invest_hat_shift')

    parameters['mse'] = mse
    parameters['rmse'] = np.sqrt(mse)
    parameters['nobs'] = model.nobs
    parameters['ssr'] = model.ssr
    parameters['t_const'] = model.tvalues['const']
    parameters['t_value'] = model.tvalues['value']
    parameters.rename(columns = {'const' : 'b0', 'value' : 'b1'}, inplace = True)
    parameters['r2_adj'] = model.rsquared_adj
    
    return parameters

grouped = df_grunfeld.groupby('firm')
df_params = grouped.apply(lambda x: invest_params(x, True))
df_grunfeld_output = df_grunfeld.join(df_params, rsuffix='_coef')

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.