3

I have a list of dates in a jason file, and like to aggregate them to see how many I have in 10 ­minutes time intervals.. I think time series in Pandas is what I should look for, but I am confused! Any idea?

[more details] when I load the json file using pd.read_json; I get only one column; which has about 10,000 rows. each row is a pandas.tslib.Timestamp such as "1970-01-01 20:12:16". Ideally, I like to group these Timestamps into 10 ­minutes time intervals; to see how many Timestamps I have in each interval and draw a bar chart (histogram).

3
  • well, how are the times formatted? what have you tried so far? you have to give us more to work with.
    – n1c9
    Commented Feb 19, 2016 at 18:45
  • 1
    Please take a look here to learn how to ask a good question: stackoverflow.com/help/mcve
    – dot.Py
    Commented Feb 19, 2016 at 18:50
  • just added more details..
    – SN65
    Commented Feb 19, 2016 at 19:28

2 Answers 2

5

You can do this doing resample.

First index on the timestamp column, if you have not already done so:

df.set_index('time', inplace=True)

Add a numeric column (you need something for resampling to aggregate):

df['count'] = 1

And finally resample as needed:

df.resample('10T', how='sum')
0
3

I use truncation to do this:

import random
import pandas as pd
import datetime as dt

ts = [dt.datetime.now() + dt.timedelta(minutes = random.randint(1000)) for _ in range(1000)] 
df = pd.DataFrame(ts, columns = ['ts'])

def truncate(t):
    return dt.datetime(month = t.month, day = t.day, year = t.year, hour = t.hour, minute = (55 / 10) * 10)

df.ts.map(truncate).value_counts()

will give you the counts per 10 minutes

2016-02-20 00:50:00    79
2016-02-19 23:50:00    75
2016-02-20 08:50:00    72
2016-02-19 21:50:00    70
...

EDIT:

The method by A. Leistra is much better, I learned something as well. It works from the setup above:

df.set_index('ts', inplace = True)
df['count'] = 1
df.resample('10T', how = sum).head()

count | ts
--- | ---   
2016-02-19 21:00:00     5
2016-02-19 21:10:00     11
2016-02-19 21:20:00     17
2016-02-19 21:30:00     13
2016-02-19 21:40:00     11
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.