21

When trying to calculate the exponential moving average (EMA) from financial data in a dataframe it seems that Pandas' ewm approach is incorrect.

The basics are well explained in the following link: http://stockcharts.com/school/doku.php?id=chart_school:technical_indicators:moving_averages

When going to Pandas explanation, the approach taken is as follows (using the "adjust" parameter as False):

   weighted_average[0] = arg[0];
   weighted_average[i] = (1-alpha) * weighted_average[i-1] + alpha * arg[i]

This in my view is incorrect. The "arg" should be (for example) the closing values, however, arg[0] is the first average (i.e. the simple average of the first series of data of the length of the period selected), but NOT the first closing value. arg[0] and arg[i] can therefore never be from the same data. Using the "min_periods" parameter does not seem to resolve this.

Can anyone explain me how (or if) Pandas can be used to properly calculate the EMA of data?

2

4 Answers 4

18

There are several ways to initialize an exponential moving average, so I wouldn't say pandas is doing it wrong, just different.

Here would be a way to calculate it like you want:

In [20]: s.head()
Out[20]: 
0    22.27
1    22.19
2    22.08
3    22.17
4    22.18
Name: Price, dtype: float64

In [21]: span = 10

In [22]: sma = s.rolling(window=span, min_periods=span).mean()[:span]

In [24]: rest = s[span:]

In [25]: pd.concat([sma, rest]).ewm(span=span, adjust=False).mean()
Out[25]: 
0           NaN
1           NaN
2           NaN
3           NaN
4           NaN
5           NaN
6           NaN
7           NaN
8           NaN
9     22.221000
10    22.208091
11    22.241165
12    22.266408
13    22.328879
14    22.516356
15    22.795200
16    22.968800
17    23.125382
18    23.275312
19    23.339801
20    23.427110
21    23.507635
22    23.533520
23    23.471062
24    23.403596
25    23.390215
26    23.261085
27    23.231797
28    23.080561
29    22.915004
Name: Price, dtype: float64
2
  • For a reason I'm not really able to explain, I needed a slightly different form of this line of code (@chrisb): pd.concat([sma, rest]).ewm(alpha=1/span, adjust=False).mean() With this, it did what I expected and took the previous average, multiplied it by (span - 1), added the new value, and divided the total by span.
    – Julian7
    Aug 17, 2019 at 19:18
  • using ewm is giving me an TypeError: unsupported operand type(s) for /: 'EWM' and 'EWM' , why is this ? how to solve it ? I'm using a windows system with spyder as ide with anaconda. my pandas version is 1.0.5
    – Aswin Babu
    Jun 28, 2020 at 19:07
11

You can compute EWMA using alpha or coefficient (span) in Pandas ewm function.

Formula for using alpha: (1 - alpha) * previous_val + alpha * current_val where alpha = 1 / period

Formula for using coeff: ((current_val - previous_val) * coeff) + previous_val where coeff = 2 / (period + 1)

Here is how you can use Pandas for computing above formulas:

con = pd.concat([df[:period][base].rolling(window=period).mean(), df[period:][base]])

if (alpha == True):
    df[target] = con.ewm(alpha=1 / period, adjust=False).mean()
else:
    df[target] = con.ewm(span=period, adjust=False).mean()
4
  • 1
    I don't understand what "base" is in your code, you probably don't need it at all. Also, would be safer to covert period to float, in particular for python 2. Otherwise good answer
    – FLab
    Oct 2, 2017 at 16:00
  • 2
    Apologies for being ambiguous. base is the base column in DataFrame on which you want to compute EWMA.
    – arkochhar
    Oct 3, 2017 at 17:34
  • small correction df[target] = con.ewm(alpha=1.0 / period, adjust=False).mean()
    – AbhijitG
    Dec 27, 2017 at 12:11
  • This is a very correct answer. People they do not realize the difference between span and alpha values. Especilly people who try to teach MACD indicator in finance on internet have wrong values. Apr 30, 2022 at 11:41
7

Here's an example of how Pandas calculates both adjusted and non-adjusted ewm:

name = 'closing'
series = pd.Series([1, 2, 3, 5, 8, 13, 21, 34], name=name).to_frame()
period = 4
alpha = 2/(1+period)

series[name+'_ewma'] = np.nan
series.loc[0, name+'_ewma'] = series[name].iloc[0]

series[name+'_ewma_adjust'] = np.nan
series.loc[0, name+'_ewma_adjust'] = series[name].iloc[0]

for i in range(1, len(series)):
    series.loc[i, name+'_ewma'] = (1-alpha) * series.loc[i-1, name+'_ewma'] + alpha * series.loc[i, name]

    ajusted_weights = np.array([(1-alpha)**(i-t) for t in range(i+1)])
    series.loc[i, name+'_ewma_adjust'] = np.sum(series.iloc[0:i+1][name].values * ajusted_weights) / ajusted_weights.sum()

print(series)
print("diff adjusted=False -> ", np.sum(series[name+'_ewma'] - series[name].ewm(span=period, adjust=False).mean()))
print("diff adjusted=True -> ", np.sum(series[name+'_ewma_adjust'] - series[name].ewm(span=period, adjust=True).mean()))

Mathematical formula can be found at https://github.com/pandas-dev/pandas/issues/8861

5

If you are calculating ewm of ewm (Like MACD formula), you will have bad results because the second and following ewm will use index starting by 0 and ending with period. I use the following solution.

sma = df['Close'].rolling(period, min_periods=period).mean()
#this variable is used to shift index by non null start minus period
idx_start = sma.isna().sum() + 1 - period
idx_end = idx_start + period
sma = sma[idx_start: idx_end]
rest = df[item][idx_end:]
ema = pd.concat([sma, rest]).ewm(span=period, adjust=False).mean()
3
  • What does item refer to?
    – tehfink
    Apr 11, 2021 at 22:38
  • replace item with "Close" to get working code. Mar 29, 2022 at 19:18
  • Or just replace all that with ema = df["Close"].ewm(alpha=2 / (period + 1)).mean() to get the same trend line in a single line of code. 2 is a smoothing value here. Mar 29, 2022 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.